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FOREWORD 


Each year national and international 
agencies spend millions of dollars on the 
preparation and production of communication 


materials related to social development 
programmes and activities. Such materials 
range from the simplest flip-chart and 


single-colour poster, to highly sophisticated 
video and 16 mm film productions in full 
colour. 


The objectives are nearly always the same: 
to use communication materials to improve the 
quality of life and wellbeing of the most 
disadvantaged groups in society, whether 
urban and rural, but always economically or 
otherwise disadvantaged. By now we know a sad 
truth that such materials, despite all the 
hard work and good intentions often fail to 
convey the messages, techniques or 
information intended. Why? The problem can 
often be traced back to the fact that not 
sufficient attention has been paid _ to 
realistic and scientifically sound pretesting 
at the community level. 


This manual on the many aspects of 
pretesting of development communication 
materials is based on nearly 10 years of 
practical work at the village level, 


primarily in Nepal, but also in Burma, and it 
is hoped that it will be of good use to those 
who are concerned with the development of 
communication material aimed at communities. 


The survival and wellbeing of young 
children can largely be controlled by mothers 
and the families. Simple preventive measures 
such as hygiene and cleanliness, and more 
complicated ones such as immunization, are 
now widely used. Simple remedies for 
diarrhoea, a major child killer, includes the 


use of oral rehydration mixtures, both 


home-made solutions and commercial packets. 
Likewise, child nutrition can be 
Significantly improved in the critical 4 to 
18 months age-period with the retention and 
protection of breastfeeding and with 
appropriate use of local weaning foods. 


In order to achieve lasting progress in 
improving child care and nutrition, however, 
the proper use of communication materials and 
techniques is essential. This pretesting 
manual is an important contribution to better 
child care. Sound pretesting of all 
communication materials will increase the 
survival of young children, especially those 
children who belong to UNICEF's target group: 
the economically, socially or geographically 
disadvantaged. 


In this context, when we are emphasising 
important behavioural changes - as opposed to 
capital and technical investments - the role 
of Project Support Communication (PSC) takes 
on added significance, and pretesting becomes 
a vital component of a strategy that seeks to 
make significant improvements for mothers and 
children - now. 


Arising from work in Nepal and 
increasingly . evident from experience in 
Burma, is the fact that for lasting change, 
people must be mobilised not merely as 
recipients and beneficiaries of development 
projects, but as active partners and shapers 


of their own destinies. 


H. K. Kuloy 
UNICEF Representative 


Rangoon, February 1984 
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INTRODUCTION 


written primarily for 
and decision-makers 
There is an 


This manual is 
planners, supervisors 
working at the project level. 
urgent need for greater emphasis on 
pretesting in development work, and middle 
level project people may be best placed to 
initiate action in this field. 

After reading this manual, you should be 
able to conduct pretesting, as well as train 
and supervise others in the methods of how to 
test educational materials. 

The manual deals with testing of printed 
materials only, since this is usually where 
projects start their work with communication 
materials. The methods can also be 
transferred to testing radio programmes - the 
principles are basically the same, though the 
techniques for testing radio programmes will 
vary slightly from the ones described here. 
We have chosen not to deal with pretesting of 
video and film, since these media play a 
relatively small role in development work so 
far, reaching as they do a very small and 
privileged group with access to electricigy 
and regular exposure to media. 

The large majority of poor people that 
most development projects try to reach, are 
without such access to electicity and media. 
However - the same rule applies to film and 
video pretesting as to radio the same 
principles as discussed in this manual will 
apply, but the techniques of testing will be 
somewhat different. 

The literature on pretesting is scarce, 
and I hope that this manual will be a useful 
addition. UNICEF has been, and still Ls ccthe 
agency that places’ great emphasis on 
developing Project Support Communication 
(PSC) materials, including pretesting. Most 
of the work this manual is built on has been 
done during my time of service in UNICEF in 
Nepal and Burma, and I am very grateful to 
UNICEF for the Opportunity to put it 
together, after nine years of working with 
pretesting in the field. 


The groundwork for pretesting was done 
with UNICEF staff and Government colleagues 
Semerepal, starting ine ero75. The bulk of 
experience and examples used in this manual 
is thus from Nepal, but My experiences in 
other countries (Sriv banka, Thailand, 
Indonesia, Kenya) indicate Similar problems 


of information gathering and comprehension o: 
visuals. A 3-month communication consultanc 
in June 1982 also allowed me to do pretestin« 
in Burma, with both, “UNICEF staff “am 
Government colleagues. Materials developex 
during this period and up till the end o: 
1983 have also been included in the manual. 

I would like to extend sincere thanks tc 
all the UNICEF staff and Government 
colleagues who have contributed to _ the 
development of this manual. In particular, 1] 
would like to thank Mr. H. K. Kuloy, UNICEE 


Representative in Burma, who took _ the 
initiative and encouraged testing chilé 
health and nutrition materials during the 
Garly days of PSC in UNICEF, and hae 


supported the activity strongly ever since. 
Samphe Lhalungpa, the PSC officer in Rangoon, 
also deserves special thanks as a colleague 
with whom the work in Burma was developed, 
and is continuing. And many thanks to Beryl 


Mackey, Swe Swe Tin and Aye Aye Shein for 
their help in typing, drawing, pasting, 
cutting and producing this manual. In Nepal, 


I would like to thank Datta Roy and Narendra 
Basnet, both in the PSC section in UNICEF. 
Datta and I experimented with pretesting 
techniques in the early days, and taught 
several courses on  pretesting together. 
Narendra joined the group with his drawings, 
and through pretesting became a very good 
educational artist. 

Finally I would like to add that for my 
own development as a communication 
Specialist, pretesting is the activity that 
has taught me the most about how people - 
both villagers and trainers - think, feel and 
react, what is important to them, and why. It 
has taught me never to assume that what I am 
doing conveys what I think it should convey, 
until I have found out - through testing - if 
it really does. This process takes more time 
and effort but in the long run is the most 
effective way to do things. 


This manual is printed for use in Burma, 
but is also intended for other countries. 
However, the first edition can be looked upon 
aS a pretest in itself, and based on 
reactions from the users, another edition 
might be prepared. Thus, we invite comments 
and suggestions - positive and negative - to 
the PSC section in UNICEF, Rangoon. 

Good testing! 


Rangoon, February 1984 
Ane Haaland 


PURPOSE OF THE MANUAL 


After reading this manual you should be 


able to: 


1. 


Posters should be pretested. 


Understand the role and the use of media 
in development projects, their 
possibilities and limitations as 
information channels, and their potential 
for changing attitudes and behavior. 


Understand the pretesting process - why, 
how, where, when and with whom to re far tie bi oe 


Comprehend the necessity for pretesting 
media at an early stage of the production 
process. Know the advantages of producing 
media that have been tested over media 
that have not been so tested 
advantages of pretesting) . 


(i.e. the 
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After reading 


this manual, you will know how to do it. 
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4. 


Plan and pretest different types of media. 
Collect sufficient information of 
acceptable quality to enable improvement 
of the form and content of the media. 


Plan and carry out a pretesting workshop, 
after having conducted several pretests in 
the field. 


Understand the need for training field 
workers in the basic principles of 
teaching and learning, thus enabling these 
workers to use the pretested visuals in an 
effective way - to broaden knowledge and 
inspire attitudinal and behavioral changes 
in the intended audience. 


Become aware of your own assumptions and 
attitudes, the symbols and style you are 
used to, the cultural basis of your logic, 
reasoning and abstract thinking, - and how 
these features of your education and 
environment may prevent you from 
communicating effectively with other 
groups with different background. 


DEFINITION 


Throughout this manual, and in many other 
communication books or articles you may read, 


you will find a number of jargon words. Below 

are some definitions: 

Message: The content of your communication 
materials - what you want’ the 
people to learn. 

Audience (or target audience): The people 


you have made your communication 
materials for. You can have primary 
and secondary audiences, e.g. a 
flipchart on rehydration solution 
has mothers and grandmothers as a 
primary audience, and fathers as a 
secondary audience. 


Respondent: The person you interview about 
your communication materials. 


Visual perception: The way people interpret 
pictures. Sometimes we also talk 


about the level of visual literacy, 
this measures how much experience 
people have in being taught with 
pictures. 
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OF TERMS 


Flipchart : 


Flashcard: 


Audio-visual: 


Visual: 


A series of pictures that ar 
bound together, usually at the top 
and can be "flipped over" by th 
teacher telling the story. Eac 
picture tells part of the story 
and together the pictures make th 
story or the message. 


Same as flipchart, except th 
pictures are not bound together 
Thus, they can be used in othe 
ways - e.g. by putting them up nex 
to each other so that the audienc 
can see the whole story together 
The cards can also be change 
around, if the original sequence o: 
the story is not clear enough. 


Communication material tha‘ 
has both sound (audio) and picture: 
(visual). This can be, for example, 
a slide show with a recorded taps 
to go with it. 


Communication material with 
pictures. Usually, we talk about 
"visual aids" ~ this means 


materials with pictures that will 
help or aid the teacher to do his 
or her work. 


The nurse is 
the 
pretester, 
the mothers 
are the 
respondents. 
They are 
members of 
the target 
audience for 
this visual. 
The message 
of the visual 
is "Eat more 
green leafy 
vegetables." 
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THE ROLE OF MEDIA IN THE 
DEVELOPMENT PROCESS 


Media are used to communicate ideas. 
Communication is a two-way process, and in 
this manual it can be defined as "the means i 
to create a shared meaning with another SA. [ie 
individual, through exchange of opinions, pulli= 
ideas or information." — 

Some people live and work under the 
impresssion that "media alone can do the job" 
in development. Experience shows that this is | 
not true. Used in isolation, media have a J — 
limited, if any, effect at all. aH TANS 

Before looking at the techniques of Wn 
pretesting educational materials, it might be \\A\) 1 
useful to look at the materials from another 
perspective: Where and how and why do they 
come into the planning process of a project? 

During the last 25 years, development 
communication philosophy has gone through 
several stages. A brief reveiw here will give 
at least some background for the philosophy 
in this manual: That projects have to start 
with where the people are, what they think 
and what they need - in other words - = 
community development. 

In the late sixties, mass media techniques 
from the West were adopted by development 
communication professionals, and used to try 
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to persuade rural people to change or adopt A few years later, the next step added 
technical innovations. The results were not extension workers with interpersonal 
impressive. communication skills to spread information to 


rural people, using visual teaching aids. 
This approach did not prove very successful 
either. In both these approaches, 
communication is viewed as a one way process, 
in which the planners and decision-makers 
hold the solution to people's problems and it 
was just a question of spreading information 
about this to the right people, then 
"development" would follow. 
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people think, and their holistic view of Tif 
as opposed to the segmented view (health 

Learning from the lack of success of these agriculture, etc.) of most developmen 
approaches, development turned to the social Projects. But stilgy people living in thes 
sciences. "Communication" - as a combination Societies or communities were not consulte: 
of the social sciences - was used to study much about the kind of development the: 
how societies function; their traditions; how wanted and needed. 
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This aspect came into focus in the next 
Stage, which took the observations from 
social scientists and other development 
workers and put it into the practice of 
community development. "Community 
development"is taken here to mean a process 


in which the planners and decision-makers 
develop a project together with th 
"beneficiaries" or "target audience". 
Communication is a vital tool in this 
process, to facilitate contact between 
planners and community members, and create 
Situations and an atmosphere where both 
groups are willing to listen to and take each 
other's advice. 
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What has this got to do with pretesting? 
If a project is developed from an office 
desk, without basis in social reality, then 
efforts to make and test communication 
materials are wasted. The best thing one can 
hope for in such a case, is that the feedback 
from the field, Saying that neither the 
project nor the materials are viable, will be 
taken seriously by the planners. 


Communication materials are only useful 
development/learning tools in a project that 
is well conceived and planned. Therefore, a 


communication person needs to know project 
planning. 
When project aims have been cleariy 


defined in cooperation with the people whom 
the project is going to benefit, a strategy 
for how these aims are going to be achieved 
needs to be worked out. Part of this is 
deciding what are the practical tasks to be 
done, who is going to do these tasks, what 
kind of background and skills these people 
have, and what they need to learn to be able 
to perform these tasks. | 


For the communication materials, the 
content of the messages need to be decided, 
based on the project aims, and on the 
socio-economic situation of the audience. 
Knowledge is needed on the kinds of media 
that are liked and understood by the 
audience, and the kind of media the users 
(e.g. health workers) are familiar with. Only 


then can the decision be made about what 
kind(s) of media should be developed. This 
Manual gives some guidelines on these 


questions. 


When making these communication materials, 
to consider whether it is 


one also needs a: 
possible to arrange workshops or training 
courses for the users/field workers’ to 


upgrade their teaching skills, or whether the 
Materials have to be developed within the 
limitations of present skills. 


An additional note is called for about 
the effect of media in the process of 
behavior change. We can summarize behavior 
change in the following stages: Awareness, 
comprehension/interest, trial/decision to 
act on new idea, action, and 
adoption/rejection. Much of development 
work is aimed at inspiring people to change 
behavior, or try new ideas or practices. The 
communication materials we develop are 
usually aimed at helping this process. 


\ 


Media is usually aimed at motivating people 
to change their behavior. But media alone 


will not accomplish this. 


2 lll Ul CU LlUll—~— 
However, behavior change is a slow and 


complicated process. Mass media can be 


effective in the first two stages, where 
people become aware of a new idea, and 
understand what it is all about. For the 


next three stages - from when they try it 
out until they decide to use the 
idea/practice permanently, or go back to 
their old behavior - personal interaction 
with people who are trusted and respected is 
essential. These interactions can be 
Supported by different kinds of media, 
however, the media alone do not get people 
to change behavior. (People who are planning 
and designing materials aimed at inspiring 
change in behavior and attitudes, would be 


wise to consult literature on this subject. 


It is a complex issue, and it is extremely 
important, but not always well understood by 
development workers) . 


In this approach to analysing needs and 
tasks, media comes into the process ata 
very late stage. This is usually where media 
belongs - as a Support to learning, not as a 
substitute to learning the real skills or 
carrying out the tasks. This approach is a 
far cry from the one which views 
communication support as a pretty poster 
stuck on a wall, and as such it takes more 
time, skills, insight and resources. 
However, it has been shown to have a 
discernable effect on the people development 


aims at helping. 
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WHAT IS PRETESTING ? 


Pretesting means field testing 
communication materials before they are 
produced or printed. By interviewing the 


audience the materials are meant for, we 


Ping out if they are understood, i.e. if the 
message or idea is conveyed in the way it 


Wace intended. We also find out if the 
audience like the materials. 

Posters, flipcharts, flashcards, 
pamphlets, handbooks, radio-programmes, 


video-programmes or film are some of the 
Materials that can be pre-tested. 

Testing e.g. a flipchart (a of 
pictures making up a story or a sequence of 


series 


WHAT DO YOU 
THINK THIS LINE 
IS? 


HA! HA! 


SN ENGARNIC 


OF COU RSE 


instructions of how to make something, e.g 
rehydration solution@ftor children wire 
diarrhoea) means taking the materials to the 
target audience when the materials are in ~ 
rough state, e.g. pencil drawings. The ide 
is to test while de etiss still possible 
Change the materials, depending on _ the 
audience's reaction. 


Pretesting may have to be done several 
Cimes.- If a flipcharieers tested, and jam 
Suggestions from the audience have been useé 
to make a new version, this version should 
also be tested as well. Test each new version 
until you have a material that the audience 
like and understand well. 
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When testing a picture with a "bubble" to 
indicate what the person says, people in 
Nepali villages thought the "bubble" was a 
garlic. They are not@famitiar with the aadaee 
of this symbol being used to show what people 
say 

What is the situation in your country? 


WHAT KIND OF 


QUESTIONS CAN 
PRETESTING ANSWER ? 


Through pretesting, you can find out how 
well you communicate with your audience in 


terms of: 


- Do they like the materials? 

- Are they used to abstraction of ideas, or 
do they mostly think in concrete terms? 

- Do they understand symbols? 

- Do they get the message right away, or are 
they confused by the way things are 
portrayed, or by unnecessary details? 

- Are they familiar with pictures in a 
series - do they connect the pictures in 
€.g. a flipchart with each other, or do 
they interpret each picture separately? 

- Do they see the relevance of the picture 
or situation portrayed, to their own lives 
and their own needs? 

- Does any part of the picture embarrass 
people? 

- How much experience do they have in being 
taught with pictures? Based on this, what 
is realistic to expect that they will 
understand? 

- What significance is attached to the 
different colours? 


When testing this picture (top) in Nepali 
villages, mothers said that the woman in the 
picture was a city woman, and had nothing to 
do with them. To them her hair style was very 
modern, and her blouse not the type worn in 
villages. The baby is also wearing city 
Clothes. And she is using a spoon, which is 
not typical. The next version (right) shows a 
typical village mother from a middle farmer 
family feeding her baby the way most 
villagers do - with her fingers. 
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WHY PRETESTING ? 


Why is it necessary to get answers to the 
above questions? Because if we don't, we may 
waste money, time and resources. And even 
worse - we may prevent the audience from 
sharing ideas that may help them improve 
their lives. We may even communicate negative 
ideas and messages. 


Communication materials are most often 
developed by urban, educated, modern, 
comparatively well-off, healthy men who are 
Surrounded by visual stimulation. While 
materials are meant most often for Yura. 


illiterate, conservative, comparatively poor, 
relatively unhealthy people - largely women - 
who live in villages with limited exposure to 
isual stimuli. 


posters, pictures or other 


This situation contributes significant: 
to communication gaps - i.e. the two grou 
(the development workers and the villagers 
see things very differently. Unless planne: 
verify their ideas and visuals with thei 
audience, things can go very wrong. There az 
sufficient examples of this in development 
There also exist projects where a dialogv 
was started between development workers an 
villagers about ideas and the communicatic 
materials - and where, as aresult, villager 
have implemented new ideas that fit thei 
environment, beliefs and culture. 


Pretesting is a cost effective way t 
prevent large, expensive disasters. If 10,00 
copies of a teaching poster are sent out t 
all the health workers in the country, an 
you find out later that villagers do no 
understand or accept the poster, you hav 
made an expensive mistake. If instead, yo 
pretest the poster, discover the mistakes 
make the changes, test it again until peopl 
understand and accept it, you can be faire 
sure that the teaching poster might be ; 
useful communication aid for the healt! 
workers. The expense of the testing is very 
small compared to the cost of a ten thousanc 
copy failure. 


This poster was tested (after production) 
in rural Nepal. Only people with more than 
five years of schooling understood the 
message correctly - these represented less 
than 10% of the respondents. 


Illiterate villagers said things like "a 


devil", "Sick ~ Eager. Pe Lind... man"| and 
"Englishman". Nobody understood the symbol. A 
few understood the pipe and the packet of 
Cigarettes. 


Red does not represent "danger" in Nepal, 
as it does in the West (e.g. traffic lights). 
On the contrary, red is a festive colour. 
Thus, both the colour and the symbol are 
Misunderstood, and the production of the 
poster is a waste of money which Pretesting 
could have prevented. | 


These pictures show three stages of tne 
Jevelopment of one of ten teaching posters in 
2 series developed by Ministry of Health and 


UNICEF, 
pretested 
villagers. 
four times 
content was found. 


Nepay sti tt. «© Te |)§6series 7 yae 
with health educators and 
Some of the posters were changed 
before an acceptable form and 
None of the posters were 
Maght" the first time. Pretesting and 
revision took about three months. These 
posters have been reprinted several times, 
and are still being used all over the 
country. It pays to spend time and energy on 
developing communication materials. 


The pretesting of the "Supplementary food 
for pregnant and lactating mothers" poster 
showed: Stage 1: The pregnant mother was not 
pregnant enough. The breast-feeding mother 
was too poor and scruffy, and she sits in an 
awkward position (used only in some parts of 
the country). Also, the baby is too old = 
there should be a picture of a mother with a 
newborn baby to give’ the idea that 
supplementary food is needed from the time of 
birth. 
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stage 2: Pregnant mother now perceived as 
more pregnant - It is the same mother, but in 
another position. The breast-feeding mother 
is liked and accepted, and the baby is 
identified as being less than a week old by 
all respondents. But, some asked, what are 
these mothers going to eat? What kind of 
advice are you going to give them? 


Stage 3: An attempt to answer this question 
saw the addition of the three food groups, 
with the advice that mothers should eat some 
food from each of the three groups every day. 
This poster was well understood and liked by 
the health educators; villagers could 
identify all the items in the poster (with 
the exception of a few who had difficulties 


identifying the foods). However, the concept 
or message waS not understood by the 
villagers on reading - this has to be taught 


by the health educators. 


(Note: It has been shown later that the idea 
of the food groups is too abstract, and does 
not relate to the villagers' way of thinking 
about food. It makes more sense to show 
typical meals with the right ingredients, and 
encourage mothers to eat such food. (This is 
one of the posters that could be revised). 


By going to the audience to ask for their 
opinions and suggestions, organizations and 
projects gain credibility. People are more 
likely to listen if they have been listened 
Poerirst. 


Pretesting sets a healthy tone for a 
project and the people they are working for - 
and with. It takes the assumption that the 
“planners know it all" and puts it where it 
belongs in the waste paper basket. This 
then opens up a dialogue with the audience or 
community, makes the planners listen, and 
brings the project managers to communicate 
with the audience. Pretesting implies a 
willingness to fit the project to the 
Situation in the field, and a concern for the 
j0b and people. It leads to a gradual 
improvement in communications and results in 
projects and materials that are more 


appropriate to communities. 
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Where the materials have been pretest 
there is a greater possibility that 
feedback will be used whether negati 
positive, or mixed. 


Can evaluation or post-testing substitt 
ror pretesting? Now eWhile it is of sees 
important to get feedback on the field use 
the materials, at this stage there is 7 
much you can do beyond learning from t 
possible mistakes that have been made. Whe 
such feedback is negative, it may be hidd 
rather than acknowledged as people are oft 
unwilling to admit mistakes. There is also 
incentive to change at this stage t 
materials are produced, and most people a 
getting on with the next job. 


RRS Ba8o8oos4l 
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WHY PROJECTS IGNORE PRETESTING 


Why then don't all projects pretest to 
make better communication materials? Is it 
because of lack of knowledge about how to 
pretest, or lack of conviction about the 
necessity to pretest? 


There are several answers to these 
guestions. We will discuss some of them here. 
While many development workers and 


planners talk about pretesting, few practice 
it. The main reason for this is probably the 
attitude that "We know what they need and 
understand", a seldom stated premise that 
underlies many development assumptions. 


The most common excuse for not pretesting 
is "lack of time and money". Communication 
materials have to be ready for a certain 
date, there is often little time to find out 
if people like them, understand them or are 
offended or embarrassed by them. Also, if 
there are problems with the materials, you 
have to revise them. There will be delays. 
The bosses will fault you for not meeting the 
deadline. They will not, however, fault you 
for not pretesting. Most people prefer to 
play safe and produce the materials fast to 
please their bosses. They may drop testing 
altogether, or do a quick test with a few 
people just to say it is done. Thus, for the 
cautious development worker unprepared to 
stick his neck out to change things, "lack of 


time and money" is an acceptable excuse. 
However, if the broader question is raised 
about success or failure of a project, 


pretesting is inescapable. The issue can be 
turned around to face the problem: How to 
convince the decision-makers that pretesting 
is necessary. 


OH! MUST BE A 
GOAT on A TREE 


WHAT DO YOU THINK 


a lack of awareness 


There is 
levels about pretesting and why it 
necessary. People may know the theory, but 
have never seen the practice, and thus remain 
sceptical about pretesting. The concept 
itself may be alien to people where the 
education system is dominated by the lecture 
method rather than the participation method 
and the idea of starting where the learner 
is. People in decision-making positions in 
such systems may feel that the whole idea of 
pretesting is unsuitable to their country. 


An underlying reason for the more openly 
stated arguments above may be the attitude 
that educated people know what villagers 
need, and how it should be presented to them. 
This attitude is widespread among educated 
elites, and produces a climate that is not 
conducive to good pretesting practices. Even 
se such people do go and pretest 
communication materials, it is more often to 
test the villagers than to test the materials 
(i.e. if the villager does not interpret 
correctly, the villager is at fault). 


This attitude is one of the most important 


reasons for project failure and for the 
audience misunderstanding communication 
materials. It is difficult to confront, as 


most people will not openly accept that they 
feel themselves to be superior to people who 
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live in villages and have Hittle ~orsena 
education. This attitude often Operates ona 
sub-conscious level, and manifests itself in 
a lack of interest in pretesting ideas or 
inaterials. 


Another related reason is the 
defensiveness of the artist and/or the 
communication materials planner who may feel 
that "people should come Up ro our level oF 


understanding, and if they do not they will 


not understand 
Therefore, 


our wonderful drawing." 
there is no need to pretest... Gr, 


they may feel that if villagers do not 
understand their drawings or matrerials, the 
villagers are criticizing their work, and 


they will not accept criticism from anybody 
below them. These reasons are not usually 
stated but they may underlie statements like 
"we don't have time", or "there is no vehicle 
to take us there" or "we don't want to stay 
in the village OVEINIGht- «etc. 
The Asian cultural norms against open 
Criticism of other people's work, especially 
of elders or one's Superiors can be another 
factor preventing pretesting. People in rural 
areaS are often not used to this kind OL 
interview either, and they may not know how 
TOmecespond at Fiysue especially if the 
interviewer is in a hurry. Lack of "good 
results" the first time may be interpreted by 
pretesters and their bosses aS a good 
argument against further work in this field, 


P 
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Many development projects concentrate c 
numbers or quantities as Criteria fa 
measuring success. If ten thousand posters 


five thousand flipcharts and ai é weekl 
radio-programme are produced, the planner 
aré satisfied. Few people ask - did th 
people. understand the communicatio 


materials? How were they used? Did they hav 
any positive effects? Did people listen t 
the radio programme? Did they follow the 
advice given to them? 


Unless questions about the quality of the 
materials are asked, pretesting does not make 
any sense. 


aS 
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VISUAL PERCEPTION 
-OR THE WAY PEOPLE SEE PICTURES 


People interpret pictures in different 
ways, depending on a number of things related 
to who they are, where they live, what they 
do, etc. Development workers and artists are 
often not aware of this. 

We cannot assume that people "understand" 
pictures. Experience from many countries 
shows that people with low literacy levels 
are not used to the picture as a teaching 
medium. In many places, they do not even know 
how to interpret a picture - i.e. they do not 
recognize the objects in the picture. 

A summary of "problem areas" in the 
understanding of pictures might help you when 
you pretest: 


a) Previous exposure to and experience 
with communication materials 


mOSt people "ieain ‘only one skill “at a 
time. If they do not know how to interpret a 
picture, they have to learn this first. They 
may have seen a picture on a wall, but do not 
know what it "means". There is not much use 
in teaching people, using e.g. a flipchart, 
if they do not understand the pictures, as 
they will use all their energy trying to work 
out what is going on in the picture, rather 
than listen to what the extension worker is 
saying. 


In Nepal, a majority of the people 
interviewed all over the country in a study 
n visual perception ("Communicating with 
Pictures in Nepal") saw pictures as a medium 
to represent an object, and not as a teaching 
medium. In Bangladesh, they found that it 
takes people a year of regular television 
viewing to get used to the medium, and be 
prepared for education through TV. 


With communication materials, it is often 
easier to detect whether or not this is a 
problem, since there is usually direct 
interaction between teacher and learners. It 
is important, however, to be aware that such 
lack of experience may represent a problem. 


People who have not been exposed to 
pictures, may not even know what is "up and 
down". In remote villages, this may often be 


the case. "Reading" pictures has to be 
learnt. 


sy 


b) Literacy. 


This aspect is obviously related to the 
one discussed above. If a person knows how to 
read and write, he or she will most often 
have been exposed to pictures of different 
kinds. However, an adult with primary 
education might have forgotten what he learnt 
in school, and will be close to illiterate, 
both verbally and visually, following years 
of disuse of his or her skills. But such 
people will usually pick up skills quicker 
than somebody who has never been to school. 


c) Social, cultural and religious 
beliefs and practices 
This is a vast subject. Obviously, we 
cannot give fair treatment to this 
complicated issue here, but a few general 
comments should be made as people's social 


Situation will determine the way they look at 
and interpret pictures, and also how they 
react to outsiders coming into their 
community to test pictures, or to teach them 
something. If your communication materials 
(and your behavior when you test them or 
teach with them) are not acceptable socially 
to the audience, you have to go back to the 
drawing board. The more you know about the 
social /situatian= through thorough field 
research, the less likely you will be to make 
mistakes in this area. 


The same rule apply to the cultural 
questions - it is equally important that the 
communication materials reflect to the extent 
possible, the cultural milieu of the audience 
and, as important, contain nothing that will 
offend people's cultural values. Culture and 
religion are closely interlinked in many 
countries, thus the two should be considered 
together. 


Rural people usually pay much more 
attention to traditional values than urban 
people. This is one reason for people who 
develop communication materials (and design 
projects) sito. gagerd against the danger of 
offending peoples' social, cultural anda 
religious traditions and values. It is often 


Suggested that in countries where people's 
through 


main experience of visuals is 


| kind of people 
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religious pictures and images, such picture: 
Should be used to educate illiterates. The 
idea often does not work, because people 
interpret pictures very literally, and do not 
connect them with anything ~“bue their 
religious origin. For example, this picture 
was interpreted by many villagers in Nepal as 
an evil spirit, but they did not link it with 
any other object portrayed, to convey the 
meaning "Hada (Brom "Communicating with 
Pictures in Nepal"). 


d) The relevance of the Siiwation/ 
problem in the communication mate- 
rials to the situation/problem of 
the audience 


People Mmeed= to be able fe identify 
themselves with a situation, and the kind of 
people that are portrayed in that situation. 
If they recognize the situation, but not the 
(who may be from a different 
nation or ethnic group), they may disregard 
the proposed solution, because "those people 
are different", and what suits them, May not 
suit us (the audience). Therefore, it is 
important to make your communication 
materials either culturally "neutral", which 
is difficuare to No, or taaes them 
Specifically for an area where the same kind 
of people live. Dress, style of houses, kind 
of draught animals used - these are aspects 
people will look for an identifying 
themselves with the situation. 


The other related aspect - which is even 
more important is the relevance of the 
roblem to the audience. If the project has 
not conducted its field research properly, it 
may end up making communication materials to 
solve problems that the audience does not 
perceive as problems; or, which have low 
priority on the community's list of what 
needs to be done to improve life. be 
however, the project is founded on Village 
reality, on what people need and want, and 
what they do already, the communication 
materials will also reflect this perspective, 
and the people in the community will be able 
to relate to them. If people see that the 
subject/problem and the situation is relevant 
to them, there will be greater interest in 
learning more about how to solve the problem. 


e) If the subject is threatening 


of development, e.g. in 
the educators have to work 


In many areas 
family planning, 


with people on sensitive issues that are 
difficult to talk about. Visuals made to 
support such projects have to be very 
carefully pretested. (See chapter; How to 


establish the social setting). People may be 
unwilling to respond to questions that they 
feel are threatening, unless the interviewer 
knows how to handle the situation. Also, as 
these issues tend to confuse and embarrass 
people, their responses or interpretation of 
the communication materials may not be what 
they really feel or mean. These issues are 
very difficult for outsiders to gather 
information on, or to pretest materials for. 
The respondents need to trust the 
interviewer, and this takes more time than 
interviewers are usually able or willing to 
spend. Local contacts can be useful in these 
Cases. It is also useful to have both male 
and female interviewers, as people tend to be 
more open with members of their own sex. 
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Only 26% of the respondents in Western 
Nepal could identify this line drawing of a 
house (compared to 91% correct answers in 
Eastern Nepal). In Western Nepal, most houses 
have flat roofs. (From "Communicating with 
pictures in Nepal"). 


f) The length of the materials 


Most people with low verbal and visual 
literacy have limited attention spans 
i.e. their concentration will gradually 
decrease. If you try to test 20 different 
pictures on a person, she may not give you 
good responses to the last ten pictures, 
because she is not used to this kind of 
"work". People's attention, both for testing 
and for teaching with visuals, is usually 


very good for the first five pictures; for 
the next five the attention level is 
acceptable, but after the tenth picture, it 


drops to a level where neither testing nor 
teaching is worth while. There are, of 
course, exceptions to this - as people become 
more used to being taught with visuals, their 
attention span will increase. However, the 
ten picture limit may be a useful rule of 
thumb when pretesting. 


g) Convenience/time of the respondent 


If your respondent is too busy, and feels 
you are "taking her time" without her seeing 
much use in what you are doing, she may not 
be interested in giving you very accurate 
answers (though she may give you some answers 
to be polite to you). She may have to prepare 
a meal,or go to the field, or look after her 
Sick baby, or do any of the other tasks that 
usually keep rural women busy through the 
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h) Colours 


Cultures often attach special meaning to 
different colours. If your communication 
materials contradict these "rules" - meg: if 
red is used for happy occasions, and you use 
1€ for “asick child - it will affect your 
development message. Find out what meanings 
the colours have, and let this guide you in 
the development of materials. 


Sometimes pictures are drawn on coloured 
paper, or drawings printed on coloured 
background. This confuses people prompting 
responses like, "why does that man have a 
green face?" or "buffaloes in our Village are 
not yellow are these sick?" 
with low visual literacy level. 


from people 
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day. This is why it is important to choos 
your time well, and to make sure th 
respondent is motivated to cooperate with yo 
on the pretest. 

Other events may be happening around yo 
that may distract the attention of th 
respondent - and this will affect the answer 
you get, unless you have managed to get he 
so interested in the task of pretesting tha 
she forgets about the environment for . 
while. 
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i) Language 


The text accompanying the visuals alsc 
needs to be tested. If people understand the 
PICEURGS Ee eDut ’ not “the language in the 


communication materials, they will still end 
up confused. The language should be Simple 
and where possible local words used, e.g. for 
diseases. Sophisticated urban words, jargon 
and specialized terms should be avoided. 


HOW TO 


This chapter deals with the practical side 
of pretesting - what you have to do before 
going to the field, when you are in the field 
and when you return. 

Study this chapter carefully before you go 
to the field. Remember, the most important 
learning takes place in the field, and the 
best way to become a good pretester is to get 
a lot of experience. The section in this 
chapter may help you analyse why you do not 
always succeed in your pretesting. 


1. PREPARATION BEFORE 
GOING TO THE FIELD 


For each of the communication materials 
you are going to test, you need to know with 
whom you are going to test (target audience), 
and what you want to find out (the objectives 
of the materials). 

The target audience can be mothers, 
fathers, grandparents, children or health 
workers, etc., or several of these groups. 

Then, you need to know what kind of effect 
the materials are expected to have on the 
audience: Is it supposed to inform people, 


instruct them in a skill, motivate them, or 
anything else? 
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PRETEST 


Most of the time, the materials will be 
designed to be used by an extension worker. 
He or she will explain the pictures to the 


audience. Thus, when your pretesting 
concentrates on the pictures only, you are 
putting them to a tough test. However, the 


better the pictures are at conveying a 
message by themseives, the more effective 
they will be as teaching tools: When the 
extension worker explains the topic, the 
audience can then give all their attention to 
the teaching, rather than trying to find out 
what the objects in the picture are. 


When you know the intended effect of the 
materials on the audience, you can formulate 
your questions. It is important to be clear 
about what you are trying to find out. For 
example, if you have a poster with a picture 
of a mother feeding her child vegetables and 
rice, construct the questions so that the 
interviewer does not stop until he or she has 
got the right answer, or has found that the 
respondent does not understand the picture 
fully. If the first question is "What do you 
see in this picture", and the respondent says 
"A mother with a child", the pretester has to 
know that he or she has to ask more 
questions. This may seem very elementary, but 
it is surprising how many pretesters stop at 
just that single question, because nobody has 
told them the purpose of the picture. 


It is important to 
discuss the 
materials before 
going to the field 
to test them out. 
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Decide how you are going to record the 
answers - will you use a fixed questionnaire, 
or basic instructions and discussions with 
the pretesters before they go to the field? 
This decision will be based on é how 
experienced your pretesters are. 


The best way to find out what works and 
what does not, is by trying out various 
methods. The rest of this chapter will give 
you some ideas to experiment with. 


How long does it take to pretest? This 
depends on how much material you have to 
pretest, and how many people you are testing 
with (see chapter,p42).Experience shows that 
it takes approximately 10-20 minutes to test 
a single poster with a respondent; a series 
Or y2Omprcuures (e.g. a £ilipeneart) will take 
about an hour. 


When estimating the time, allow time fc 
finding respondents, establishing rapport 
and for refusals. A lot of time may be waste 
trying to talk with people who in the end c 
not want to be interviewed, but are curiot 
about what the pretesters are doing. You wil 
havewa better idea of Nowe 1c. Galculate see 
time after conducting and observing son 
tests with different kinds of materials i 


the field. 


examples of how to determine th 
what /ovtom 


some 
purpose of the picture, Wor 
intended to convey: 
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This picture intends to convey: Sick man 
sitting on a bed, suffering from fever. 


This series of 
pictures of a 
malnourished child 
should convey the 
following meaning: A 
very malnourished 
Charo about two 


years old (1), being 
fed soft food by her 


mother (2) and 
becoming well again 
(302 (From the 
health education 
poster series, 


Nepal). 
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9. IN THE FIELD 


This section will give you advice on 
interviewing techniques, and on the possible 
effect of your attitudes and behavior on the 
results of your work. 

In the field, the first step is to contact 
the local leaders and explain what you are 
doing, and why. Explain that you are testing 
the materials, not the villagers. Also 
explain that you want the _ villagers' 
suggestions for improving the materials. 

The local leaders can be helpful in 
suggesting where in the village you should go 
to find the kind of people you have decided 
to interview. Often, a villager leader will 
suggest that he should accompany you to help 
an your work. In most cases, this is not a 
good idea. Tactfully refuse his offer. When 
leaders come to join in pretesting, they will 
often “take over" and try to “help” a 
respondent who does not understand your 
communication materials. The village leader 
will want his people to appear "good" and 
"intelligent", by giving the correct answers 
to your questions. Villagers, and especially 


village mothers, also have a tendency to 
become reticent if the local leader is 
present - they may also be nervous to make 


mistakes in front of their leader. 


ils 
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These reasons cannot be explained to the 
leader - find some other way to dissuade him 
from joining you. Say that your pretesters 
become nervous if there are people watching 
them, and experience from other places has 
Shown that you get better results on your 
own. If he is really interested in your work, 
offer to come and tell him what you found out 
when you have finished testing. 

Sometimes the leader will insist on 
accompanying you and then proceed to disturb 
interviews and make the testing useless. In 
such cases, carry out a few interviews, and 
then leave the village. It might upset your 
schedule, but there is nothing to be done 
about it, but to find another village for 
your pretesting. 


A. INTERVIEW TECHNIQUES 


a) Establish the social setting 


Where you ask the questions is almost as 
important as how you ask them. Try to find a 
place where you will not be disturbed by 
other people. This is difficult in a village 
Situation where everybody will be curious 
about what you are doing. However, most 
respondents will feel inhibited in a crowd. 
You will also have trouble with other people 
"helping" the one who is being interviewed. 
It is worth some time and effort finding the 
best place for the interview. 
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If you sit down in a public place, or even 
outside a person's house, you will most 
certainly have a problem. The same will 


happen if you interview women in the market 
place or by the water tap. Such places are 
fine for group interviews. For individual 


interviews the inside of a house or in the 
backyard is best. 


However, always be careful to observe the 
local customs. In some places, it is not 
acceptable for a male interviewer to be alone 
with an unchaperoned female respondent. In 
such cases, an acceptable way out is to 
interview a pair of women. It is better to 
get two women to interview, than let a 
woman's husband or father-in-law be present 
at the pretesting. A female interviewer will 
not have this problem. Therefore, train both 
male and female pretesters, and share the 
work between them. A woman will have an 
easier time testing nutrition materials with 
mothers, while male pretesters will have more 
credibility with farmers. 


Children can often be difficult to deal with, 
even if you manage to establish a reasonably 
private setting. One way to deal with older 
children is to carry some comic books to lend 
them while you do your pretesting. Another 
strategy is to carry paper and coloured pens. 
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Some people insist on watching’ the 
interview, regardless of your plea for 
privacy. You can often persuade them to stay 
away by telling them you will interview them 
afterwards, and that they cannot watch this 
interview if their testing is to be valid. 
This approach can be used even if the person 
wanting to be interviewed is not actually in 
your target audience. interview him or her 
briefly - to satisfy them. 


b) Introduction: Establish rapport 


The introduction to the pretesting interview 
is very important as it will set the tone for 


the discussion with your respondent by 
motivating him or her to give you their time 
and opinions about your communication 
materials. 


Motivate them so that they see the need 
and usefulness of what you are asking them to 
do. If you ask them to agree to the interview 
for the purpose of helping you and your 
project, some might agree out of politeness. 
However, if you can make them feel that you 
are asking them because they are the 
"experts" on the subject you are testing - 
they know e.g. what their babies suffer from 
and how they could learn new techniques, they 
may listen to you more attentively. If yot 
can also make them see how their suggestions 
can improve the materials so that the 
materials become good teaching tools to help 
them and their neighbours learn more easily, 
they will most probably agree to be 
interviewed, and take an interest in the task 
as well. Your guideline should be: Most 
villagers relate to things and people that 
are close to them in distance and life-style. 
If you want them to cooperate, you have to 


relate to those things rather than to 
concepts and projects outside their 
experience. 


Even where people cannot relate to your 
project directly, say who you are, whom you 
work for, and what your project is doing 
(briefly). Invite the respondent to ask 
questions. Many people are curious about 
things you may not even have considered an 
issue, and will not ask unless invited to do 
so. Very often, you will learn interesting 
things about what people think and their 
concerns when you ask them to bring forward 
their questions. 


AMI ASAI 


A word of caution is called for here: Be 
careful not to make promises and raise 
expectations about what will be done by your 
meoject for people in this village. It is 
often tempting to make promises, thinking 
this will ensure better cooperation from the 
villagers. However, such false promises (even 
if they are well intentioned) will make 
people more sceptical to development, and 
probably prevent their cooperation with the 
next pretester or researcher who comes to 
this village. 


The normal "small-talk" also has its place 
in the introduction - do not’ spout a 
monologue. Ask about the family, about 
village matters, about the weather, etc. - 
depending on the situation and the person you 
are talking with. Be friendly to the children 
paeespecially if youw° are trying to get a 
mother's cooperation 


This part of the introduction is important 
- it will make the respondent feel that you 
are really interested in talking with him or 
her, and that you have come to their house 
because they are exactly the kind of person 
you are looking for. If they are to receive 
that "message" from you, you have to let them 
talk, too. 


You should avoid sensitive or potentially 
sensitive issues at this stage, Fe 
questions like "do you read and write ’ 
unless such information is essential to 
whether or not you are going to interview the 
person. If this information is necessary, 
explain why you are asking the question. 
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An example of an introduction could be 
something like: "Good morning, I am 
from the project in 
Rangoon. We are trying to improve the health 
of people in your village, and in many other 
villages like this in the country. We have 
developed some teaching materials for health 
workers. Are you interested in looking at 
them?...(the mother says yes, she is)... We 
do not know if these materials we have made 
are suitable for this village, and if you and 
your neighbours will like them. Therefore we 
have come to ask your opinion about these 
materials, and to learn how you think they 
could be changed so people here in thevillage 
will like and understand them better. If the 
materials are easy to understand, the health 
workers can use them to discuss health 
problems with people like you and advise on 
how to deal with the problem. 


Would you have time to discuss these 
materials with us? the woman asks .. why 
me? She says she does not know anything, 


cannot read and write, ask someone else. 


eee 


"We want to talk with you because you 
a mother, and care for the children in the 
family of the time. It is important that you 
and other mothers here like the materials and 
understand them. It does not matter that you 
cannot read and write. Look at the pictures 
and tell us what you see there - that is all 
you have to do." (... the mother is now 
convinced that you really want to talk with 
her, and not with educated women only. Talk 
with them about the family, ask if any of the 
children are sick, etc., if the mother has 
any questions. After an introduction like 
this, which may take 10-20 minutes, you are 
ready to start your interview). 


are 


c) Let people touch the materials 


If you want people to respond freely, let 
them do what they want with the materials 
(except tearing them up..). Let people touch 
and hold them. Do not behave like a school 
teacher keeping the children's fingers off 
the precious posters. Communication materials 
for field testing should be in a rough state 
- and tattered by the time you have finished 


testing. Make one or two photo copies for 
Spare copies if the materials get rough 
handling. 

Don't treat your respondents like 
children. Remember you are testing the 
materials, not the people. If you want good 


results, treat people with respect. 


2) 


d) Encourage people to talk 


Try to "step into the shoes" of the villagers 


you are interviewing. This entails a 
different way of looking at time, for 
instance. Put the city pace behind you, be 
patient, and accept that the interview may 


take a long time. It is better to take time 
and get one good interview than to hurry and 
get five bad ones. 


When testing, make sure 
that people can see the 
materials properly. If 
the pretester does not 
know her materials well, 
she may turn them towards 
herself rather than 
towards the audience. If 
people are shy, they may 
not ask for a good look 
at the material and their 
lack of response will be 
interpreted as “not 
understanding” 


Most people will never have been asked to 


comment on educational pictures before, and 
what you may interpret as lack of 
understanding or reticence, may be just 


hesitation in the face of a new situation. 
Make them feel unhurried. Let them know that 
it is perfectly normal to have trouble with 
the materials. 


Some golden rules in pretesting: Never 
make your respondent feel stupid. Do not 
argue with or contradict what the respondent 
Says. Do not interrupt. let the respondent 
talk. And do not let them feel they have said 
something wrong. 


Do not judge people - you are there to ask 
them how they interpret your materials. If 
they say that a house looks like a cow, 
that's OK. Ask them to point to the picture 
and explain what they see, and how they see 
it. If you laugh at them, and they feel that 
they have said something wrong, you may not 
get another word out of them, and lose your 
chance of finding out why there is a problem 
with the materials. 


Be neutral - encourage people to talk, to 
expand on their statements, to explain how 
they see things and why they see it this way. 
Do not show your feelings and Opinions - it 
is their feelings and opinions you are after. 
This "15°" qeereal ‘art, land@iae requires 
sensitivity and tact. 


When pretesting, resist the temptation to 
teach. You are collecting information, and if 
you do a good job, it will help those whose 
job it is to teach. Many pretesters fal] into 
the temptation of teaching. Remember that 
teaching new health practices is not 
something you do in ten minutes in the middle 
of an interview, in a village where nobody 
knows you or has a reason to trust you. 


However, this is of course, like most of 
the “rules" in this manual, just a rule of 
thumb. If you are in a house, interviewing a 


mother on a flipchart about rehydration 
solution, and you see that her child has 
diarrhoea, you can of course ask her if she 


thinks it may be an idea to try it on the 
child. However, this discussion should be 
held after you have finished testing the 
pictures in your flipchart. Sometimes the 
mother will also ask questions herself, 
because what she has seen in the visual is 
relevant to the situation in her house. In 
such cases discuss it with her, but also bear 
in mind your limitations: you are leaving. 
Try to put her in touch with the local health 
worker, or ask her where she goes for medical 
help for her children. If you sense that she 
is not going to seek the health worker, and 


that the child indeed needs help, or 
follow-up on what you have taught the mother, 
make the health worker aware oof the 
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Situation, if possible.’ These situations are 
tricky, though, as you can end up spending 
most of your time helping the sick children 
rather than testing materials. Always try to 
balance, and most important - bear in mind 
the long term solution as well. You might be 
giving more help by alerting the right people 
to the situation, than by trying to deal with 
it yourself, and forgetting about your 
pretesting. 

The better you know your task, techniques 
and materials, the more you will be able to 
act naturally and to concentrate on the 
respondent and his or her reaction and 
feelings - both stated and unstated. 

If it appears that your respondent is not 
cooperative, or that he or she does not 
understand the question, start again with an 
easier question. Always start with something 
easy to give people confidence, and then 
proceed gradually to the difficult questions. 


e) Take a few materials only 


When pretesting, it is advisable to take only 
one or two different materials. If you take 
too many, you will have difficulties keeping 
the reactions separate. Try not to test more 
than one material per respondent, unless it 
is a very simple one (e.g. a poster with one 
picture). In Nepal and some African 
countries, it was found that the attention 
span of illiterate people is approximately 
ten pictures. Their attention is very good 
for the first five, and acceptable up to ten, 
but after that it dzops rapidly. Thus, 736 
flipchart or flashcard should normally have 
no more than ten pictures, preferably less. 
So if you are testing two different 
flashcards with illiterate people, it may be 


a good idea to test only one of them with 
each respondent. 


f) Different types of questions 


There are two main types of questions people 
usually ask in interviews: Open-ended, and 
leading questions. Pretesting uses open-ended 
questions, with only few exceptions. The way 
you ask the question will determine the 
answer. Become familiar with the different 
types of questions, and "listen" to yourself 
when you are conducting the interview. 


Many inexperienced interviewers tend to 
ask leading questions, because these are 
easier to get answers to. The more training 
and experience a pretester has, the easier it 
will be to deal with open-ended questions, 
and thus get better results. 

Open-ended questions are 


asked to get 


people to express what they think, without 
providing a lead or clue to what the answer 
might be. 


Examples of open-ended questions that can 
be asked for this picture (and most other 
Pictures) : 


- What do you see in this picture? 


- What do you think this is (or could be)? 

- How do you think this person feels Cra tie 
respondent has identified the person 
already) ? 
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It is very common when asking ~—£6 
information to pose questions like "Do yo 
think this is a healthy woman?" or "Do yor 
tinkeetnis is a village?" The danger i 
asking such questions, is that one cannot 
discover what the respondent thinks. The 
respondent has been led to the answer we want 
them to give - usually a confirmation of ou 
own opinion about the picture. Questions like 
this are called leading questions. Avoid suct 
questions. 


Leading questions can normally be answerec 
by ayes" or "no". Examples: Do you think this 
man is working? Do you like this picture? Dc 
you think this man suffers from goitre? 

Another type of leading questions limits 
the respondent two options, neither of which 
are the respondent's. 

Examples: Is this woman healthy or sick? 
Is this a friend or a road? Is this family 
rich or poor? 


Leading questions should be avoided in 


pretesting. 
To collect useful information for 
improving communication materials, ask 


open-ended questions most of the time. This 
isegetricuit. People you interview will most 
often be unfamiliar with being interviewed, 
and also with the kind of materials you are 
discussing with them. Therefore, they may 
hesitate to answer at Paesce It is very 
tempting to ask leading questions to "fill 


the gaps", instead of Waiting for them to 
answer. It is also possible that they have 
Simply not understood the question, but do 


not want to ask you to repeat it. . 


Each situation has to be tackled on a case 
by case basis, but some general suggestions 
can be made: Ask "leading" questions that 
have nothing to do with the actual 
information you are seeking, just to loosen 
up the situation. For instance, you can ask 
“Have you seen anything like this before?" 
which may also give you a clue as to whether 
or not the respondent recognizes the 
situation as having something to do with him 
or her, or with the village. If he or she has 
not seen anything like the picture, 
answer is going to be pure guesswork. 


their 


Another rephrasing of the usual "What do 
you see in this picture?" is "What do you 
think this could be?" or "what do you think 
this looks like?" If a respondent has not 
understood the question, try to rephrase it 
until you are sure that he or she has 
understood. This is especially important for 
sensitive or threatening issues as 
respondents tend to react defensively in the 
beginning, and may hear something different 
from what you meant to ask. 


An example from Nepal: A survey was 
conducted to find out how much women knew 
about abortion. They were asked "Have you 
heard about abortion?" However, when the 
results (which showed that only 5% said "yes" 
to the question) were checked by people in 
whom the women had confidence, it turned out 
that most of them had heard the question as 
"Have you had an abortion?", which is illegal 
in Nepal. Almost everybody (98%) had heard 
about abortion, which is common knowledge and 
practice among villagers. (From "The use and 
Misuse of social science research in Nepal", 
by G. Campbell, L. Stone and R. Shrestha. ) 


as 


g) Probing - or follow-up questions 


several 
a 


Sometimes it is necessary to ask 
questions about a picture before 
satisfactory interpretation - or lack of such 
- of the picture can be obtained. 


Probing essentially continues the posing of 
Open ended questions and follows up on 
respondents’ responses. An example will 
illustrate how it can be done. 


ay 


You have a picture of a child lifting a 
pair of weights, encircled by different kinds 


of foods that will make him strong. An 
acceptable interpretation of this picture 
could be e.g. "If your child eats rice, 
vegetables, potatoes, sweets and fruit-juice, 
he will be strong." The interview with 
probing: 


Interviewer: What do you see in this picture? 


Respondent: (hesitates).. I don't quite know. 
I see a child; I think. 

I: That is fine. How is the child? 

R: He looks very fat. He is smiling. 

I: OK. Do you see anything else? 

R: There 1S-6-gecel1e.. . 

I: What is in the bottle? 

R: I don't know. Water, maybe. 

I: OK. Anything else? 

R: Those round things... are they eggs or 


potatoes? 


I: Well, what do they look like to you? 

R: I think they could be eggs. 

I: Fine. Do you see anything else? 

R: There are some bricks, and some small 
black spots. | 

I: What do you think the black spots could 
be? 

R: I don't know. Stones, maybe. What are 
they? 

I: Well, I don't know either. Just tell me 
what they look like to you. 

R: I think they are stones. 

I: OK. Anything else? 

R: Yes, there is a plate of rice. And some 
COZ s 
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I: That is fine. Now, looking at all these 
parts together, what does the picture mean 
to you? 

R: It is a child and.some food around it. The 
child is too small to lift those kind of 
weights. Why are they put there? 

I: I don't know. Maybe the artist was trying 
to show something. 

R: Well, I don't know what he is trying to 
show. I can see those vegetables and eggs 
and the rice, and a child lifting weights. 
What else is there to see? Everybody knows 
that children don't lift such weights. 

i= Yes, thateseeprobpably right. Now;stet. us 
go on to the next picture... 


A good interviewer who knows what he is 


after (i.e. he or she has defined carefully 
the objective of the communication 
materials), can continue to ask questions 


like this until he or she gets to the "heart" 
of the matter. Since every respondent will 
answer in different ways, it is difficult to 
construct a questionnaire that gives 
guidelines on what to ask, and at the same 
time gives you freedom to probe in the way 
described above - depending on what the 
respondent 


says. 


Good probing is aifficule in the 
beginning, but it is essentially just a 
matter of knowing how to do it, and then 


getting the experience. 


h) Giving clues 


What is the acceptable level of help to a 
respondent who has trouble with a picture? 
This is a dreiicult question, Dut ge ces ting 
promotion materials that are supposed to work 
on their own (e.g. a poster on breast-feeding 
that will be placed in offices and shops), 
you should not give any clues - or only very 
minimal ones. If you are testing teaching 
materials, it is different, because these 
will be explained by extension workers to the 
audience. Thus, the picture is not expected 
to work on its own. 


Thus, when testing teaching materials, you 
can help people to some extent, e.g. by 
pointing to the different parts and asking 
specific questions (see picture). The 
questions are still open-ended. 


You can also ask questions about whether 
there is a connection between the different 
parts of the ‘picture, and ask "them © ‘to 
describe this connection and what it may 
possibly mean. 


However, give as few clues as possible. 
The idea is to be neutral, and encourage the 
respondent to tell it as they see it. If a 
respondent is completely "lost", though, you 
should give him or her some clue to get 
started. The respondent may not know what you 
expect him to say and do, despite your 
explanation. This problem will increase when 
you go to remote villages that have not had 
much exposure to visuals of any kind. 


When you give clues, always note (or have 
your recorder or co-worker note) that a clue 
was given. This makes it easier to analyse. 
If 50% of the respondents needed clues before 
they could interpret the picture, you most 
probably will have to make a new and clearer 
picture. 


a 


ee 
ee 
ee ee 


picture): 


Example of giving clues (see 

Interviewer: What do you see in this picture? 

Respondent: I don't know. Is that a table? 
(pointing to the right hand 
corner, supposed to show a growth 
chart). 

iz: Well, it could be a table. Do you see 

anything else? 

R: A big clock, I think. 

I: Anything else? 

R 

I 


: Some blobs, funny shapes. 
: (pointing to one of the nurses) What could 
this be? 
R: Well, it is not a person, even though it 


looks like one. 

I: Why could it not be a person? 

R: It has got no legs. 

I: How about this one - and this one 
(pointing to the other nurse, and the baby 
in the weight). 


R: They look more like people. But not quite. 
We don't have any people like that in our 
village. 

..- (and the interview continues)... 


Pointing to things will often focus 
people's attention, and they can slowly make 
out the whole picture after identifying all 
the different parts. However, one should not 
take for granted that they will put the 
things together even if they identify the 
parts correctly - in places like Nepal, for 
instance, many people do not put the 
different parts together (see chapter on how 
people interpret pictures). Find out what the 
situation is in your locality - for different 
groups with different kinds of background. 
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i) Thanking the respondent for his/her 
time 

Always let a respondent know that he or she 
has been of help. Tell him or her again what 
you are going to use the information for, and 
how his/her response will help to improve the 
materials. Never abuse people's time or their 
Willingness to help. Think about future 
researchers who may come to the same place 
later. Researchers are getting a poor 
reputation in many developing countries for 
coming into a community, taking people's 
time, and never giving anything back. The 
villagers never hear or see the results of 
those long hours they invested in answering 
questions. After a few such experiences, 
people are not very willing to give you - or 
other researchers - their time. In this 
matter as in health - prevention is better 
than cure! 


J) Recording 


Pretesters should work in pairs, it at all 
possible. One person to conduct the 
interview, the other to write down the 
questions and the answers. The most important 
thing is to record the answers properly, but 


if the questions are also written down, this 


ARE HOPELESS 


[/ 


can be a good tool to improve the 
interviewer's techniques. Also, by writin 
down the questions, you will get a bette 
idea of how much you can trust the results o: 


the interview - if several leading question: 
were asked, and sensitive issues were raise 
at an early stage, the results of the 


interview have to be analysed very carefully. 
The answers may not be valid. If only answer: 


were recorded, these mistakes may be 
adifficult to detect. Yourvanalysis. of th 
results may thus be based on- wrone 
information. 

If the artist who has developed _ the 


communication materials is also pretesting 
he should be the recorder rather than the 
interviewer. Experience has shown that it is 
diffieuit for an artist’ to detach himser 
from his product, and conduct a good pretest. 
It is very common for an artist to get 
impatient with those who do not understan 
pictures. He will learn more from watchinc 
and listening the first few times. Later, 


when he discovers that he can become a better 
educational artist if he learns how people 


actually see his pictures, he can pretest his 
own materials. 


We have discussed the difficulties about 
constructing a  pretesting questionnaire, 
because it gives you less freedom in probing. 
However, it is a good idea to write out some 
of the basic questions when planning the 
interview. With such a list the recorder can 
remind the interviewer if some major points 
are forgotten. 


When using this approach, the interviewer 
is freer to ask the questions he remembers, 
and probe when he needs to, without having 
the paper in front of him. He can be certain 
that nothing will be forgotten, because the 
recorder will remind him. 


With this exception, the recorder should 
not interfere in the interview and stay in 
the background as much as possible. People 
tend to get nervous if what they say is 
written down, and they will be more careful 
about their answers. If the interviewer 
Manages to keep their full attention on the 
pictures, they may become absorbed enough to 


forget about the recorder. The pretester 
should- also explain to the respondent what 
the recorder is doing, and why it is 


important to write down his or her answers. 
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If you have to conduct the pretesting 
alone, you have two options for recording the 


answers: 1) While you are interviewing, or 
2) After the interview. 
If you write while talking to the 


respondent, you will disrupt the flow of the 
interview, and probably inhibit spontaneous 
reactions. This is: generally not recommended. 


In writing down the results of the 
interview after the testing, there is a 
danger that you will introduce your own bias 
and interpretations into the results. 
However, this method is generally to be 
preferred if you have to test alone. When 
using this method, write down the answers as 
accurately as possible immediately after the 
interview is finished. 


Some people prefer to record the 
information on a form when they are in the 
field, to make it easier to analyse the 
results. An example of such a form which was 
developed and used in Nepal: 


PRETESTING REPORT 


Title of visual: Breast is best 


Type of visual: Poster 
Target audience (primary, secondary): 


Stage of progress: 2nd 


pi Ethnic 
Sex ace group 


Educa- 
tion 


La 


50 


Visuals 


6) F Pegu Karen Mother breast- 
feeding. 
She is smiling 
Baby is healthy 
4 F Pegu Karen Mother breast- 
feeding 
She wears city 
clothes 
‘Hairstyle very 
modern 
Does not hold —— 
baby right Sr CEL, 


Mothers, grandmothers (urban) 


Interpretation 
Remarks 
Text 


Gave 
positive 
feeling 


Cannot 
understand 


Breast is 
best 


Bie 


B. TESTING WITH THE USERS OF 
COMMUNICATION MATERIALS 


When testing materials with the users, 
e.g. field workers in health or agriculture, 
you are usually testing two different things: 
1) The teaching skills of the users, and 
2) The comprehension of the pictures by the 
audience (usually the villagers). 


In this section, these two aspects of 
testing with the users are discussed 
separately. This is because the results of 
the testing requires different kinds of 
action: When you test teaching skills, you 
may recommend training courses for the field 
workers on more effective use of visual aids. 
When you test for the comprehension of the 
pictures, you may recommend changes in the 


communication materials. 


Test and discuss the materials with the field 
workers. 


,Testing the teaching skills of the users 
Should preferably be done in cooperation with 
a trainer who is involved in their programme. 
You should find out what kind of skills the 
field workers have, and what kind of skills 
they need. In most cases, the design of your 
_communication materials has to fit with the 
present level of the field workers' skills. 
For the long term improvement of the skills, 
you may be able to recommend to the training 
section in the project that they include 
sessions on teaching with visuals in their 
next’ seminar or workshop. 


» 


When assessing the teaching skills of the 
field workers with the objective to design é 
training course, you should draw up _ some 
guidelines of what to look for. If you have 
to do this assessment by yourself, you coulc 
get some guidelines from an experiencec 
trainer. Here are some suggestions about what 


to Look tor: 


- Does the field worker involve the audience 


in a discussion about the subject by 
asking them questions, or does he give 
them a straight lecture? 

- Is he sensitive to whether or not the 
people understand what he is’ talking 
about, and does he know how much they know 
about the subject? 

- Does he relate the situation in the 


picture to this particular village? 

- Does he give people enough time with each 
picture? 

- Is he aware that some pictures may be 
difficult to understand, and does he take 
special care in explaining and discussing 
these pictures? 

- Is he showing respect towards the people 
he teaches? 

- Does he appear to have good knowledge of 
and tolerance for local customs, beliefs 
and practices in health matters? 


You" May also check® 7@he “section “on 
"Attitudes and behavior", as the issues 
discussed there are as important for teaching 
as they are for information gathering. 

When you are assessing the teaching skills 
of the field workers with the objective of 
fitting the communication materials to their 


present skills, look for the following 
points: 
- Are they familiar “withietne. type Son 


materials you are testing (e.g. have they 
used a flipchart before?) 
- Do they show the picture clearly to the 


audience? 

- Do they take enough time with each 
picture? 

- Do they like using the communication 
INaterials (i.e... is ditwemkely that they 


will use something you send to them, or do 
they regard the materials as an extra 
burden) ? 


Such an assessment can usually be made by 
watching field workers teach villagers in an 
environment they are comfortable in. 


Some field workers have 
little or no experience 
jin teaching with visuals. 
They need training before 
they can make good use of 
the communication 
materials. A good way to 
train field workers is by 


role play: Let two 
participants be health 
worker and mother, have 


the group or class watch, 
and discuss what was good 
and what needs to be 
improved in the teaching 
methods. 


If the results of the testing show that 
the field workers have little or no knowledge 
about how to teach with visuals, it may be 
best to convince the training institution 
that a training course is needed. To produce 
materials for inexperienced field workers can 
be a waste of time. 


field workers on how to 
and why it should be 


Instruct the 
pretest the materials, 
done. To begin with, ask them to use the 
materials in the same way they would 
normally. Discuss why you are asking them to 
use the materials - you want to design 
teaching aids that are useful to them in the 
field, and to do this well, you need their 
advice. Explain that you are also testing the 
materials with the villagers, and show them 
how you are conducting such a pretest, if 
possible. Also tell them that you will have a 
discussion with them after the field workers 
have tested the materials, to get their 
suggestions for improvement of the form and 
the content of your materials. 


The same pre-testing principles apply 
here: You are testing the effect of the 
materials, not the intelligence of the 
audience. This should be made clear to the 
people you are testing with after the field 
worker's session. Explain why it is important 
that they tell you exactly what they think of 
the materials, and if they nave any 
suggestions for improvements. Also ask them 


what they think about the way the materials 
were being explained by the field worker. You 
might find out something about how people 
learn in this community. 


These are some general guidelines for the 
types of questions you could ask, depending 
on the materials you are testing. Remember 
always that preparation is the key word - sit 
down and think through what you are going to 
ask people, whether it is the villagers, the 
field workers, the audience of the field 
workers, or the trainers. For each issue, you 
have to decide your purpose, and then design 
the questions. If you follow this method, and 
combine it with an open mind towards adapting 
to the situation (rather than following a: 
rigid questionnaire only), your projects and 
communication materials will benefit greatly. 


In addition, check with some of the 
audience what they have understood after 
being taught by the field worker using these 
materials, and if they have any suggestions 
for improvement. You can pick out a few 
people and ask them to talk to you after the 
field worker has given his or her lesson, 
using the materials. This task is easier if 
there are two or three persons conducting the 
testing. You should try to find a place with 
reasonable privacy for these interviews, e.g. 
a room, if you are in a health post, or a 
private corner. 


34 


C. BEHAVIOUR AND ATTITUDES 


Most people are not aware of their 
attitudes, or about how their behavior can 
affect others. The purpose of this manual is 
to make you aware of how certain kinds of 
behavior can affect the results of your 
pretesting, and to help you understand why 
[his 1S So. 


The pretester who usually gets the best 
results is the one who is really interested 
in his or ‘her work, and curious about 
people's reactions to ideas and communication 
materials. He or she cares about the work, 
and would like to improve the quality of the 
materials the project is developing. Such a 
person is not defensive - he wants to find 
the mistakes rather than hide them. He will 
not have his "feelings" in the materials - 
i.e. he has not made up his mind that these 
materials are perfect. He has an ‘open mind, 
he wants to find out what people really think 
about the materials - and why they think this 
way. 


How can you learn more about attitudes and 
behavior? The subject is vast, and in this 
manual it is only possible to touch on it 
very superficially. It is a very important 
subject, 


and does not usually get enough 


attention in development work. Those who are 
dealing with training of field workers, 
researchers, etc. would be advised to look 
deeper into the subject. 


Analysing your own attitudes and behavior 
is difficult. It8 as even more difficuie. 2. 
have some one else make you aware of how your 
behavior might affect others in a negative 
way. Most people are defensive about their 
attitudes, because these are so closely 
linked to emotions. It is much easier to 
train people in the technical aspects of the 
work - in this case interviewing techniques - 
because these are skills that most people can 
learn if they want to. Attitudes and behavior 
are strong parts of your personality. 


The way people conduct pretesting will 
often tell a lot about their attitudes +o 
their work and to villagers. Many pretesters 
will not be aware of this, because most 
people do not consciously analyse their 
attitudes unless they are put in a position 
where they have to do so. Pretesting is one 
such situation - if the attitudes of the 
pretester do not fit the job he or she is 
doing, the results of the test will be of no 
use. People will not open up and respond to 
the pretester in such a way that is needed 
for him or her to get useful information from 
which the materials can be improved. 


Not many 
pretesters 
will display 
such an 
extreme 
behavior, but 
even the more 


subtle 
attitude to 
villagers' 
lack of 
understanding 
can damage 
the 


pretesting. 


Attitudes guide the way you behave. People 
will usually sense from the way you behave 
whether your attitudes to them and to what 
you are doing are positive or negative. If 
your attitudes are positive,they will most 
often respond positively. Thus , it is 
important to be aware of the attitudes in 
yourself as well as in the people you work 
with. Learning about attitudes and behavior 
is a long and slow process, but it is worth 
the time invested in the subject: Good 
communicators and trainers know that 
attitudes and behavior are equally important 
to technical knowledge about a subject. 


Let us look at some examples. A pretester 
works in an office. He usually lives in a 
city. He has good formal education - whether 
he is a planner, an artist or a researcher. 
He is comparatively healthy, and so are his 
children. When he comes to a village, he will 
test his materials with people who are mostly 


farmers, who do not have much _ formal 
education, and whose conditions of life are 
different. It is common for the pretester to 


feel that he is superior to them, that they 
are ignorant and even stupid. If he is not 
used to being in a village, he may feel 
awkward about the food, dust and other 
things. 
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If people feel that the pretester is sincere 
in his/her task, they will cooperate, and you 
can find out what they really think about the 
communication materials. 
enters SS 
If the pretester does not feel at ease, he 
will not make his respondents feel at ease. 
If they feel he is looking down on them and 
on their situation, they will not be 
motivated to work on the pictures with him. 
If he does not want to sit down on their 
porch, they may feel awkward. If one of the 
babies crawl over and wants to sit on his 
lap, or wants to look at the pictures, and he 
pushes it away, he will make the mother feel 
bad, and her attention will shift from the 
pictures to the baby. If the pretester has 
behaved with obvious disgust, her attention 
may never return to the pictures. 


Another way of alienating respondents is 
by the pretester's reactions to the answers. 
If he laughs at a woman when she says a house 
looks like a mountain, or a cow is an 
elephant, or a sick man is a bride, she will 
be offended and feel insecure. Even if the 
pretester does not show it as openly, a 
respondent can read from facial expression or 
other kinds of non-verbal behavior , that he 
thinks the respondent is stupid. 
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they will 
With respect, an open mind and 
curiosity the pretester is well equipped to 
start his or her work. 


If you respect the villagers, 
respect you. 


If the pretester asks a lot of leading 
questions, this may be related to his 
attitudes to the job. He may have set ideas 
about what the villagers will say or should 
say, and may therefore not ask them their 
real opinion about the pictures. Or - he may 
feel that it is rather pointless to ask them 
about the pictures, because he himself knows 
what they will understand. It may, on the 
other hand, also be simply a matter of lack 
of experience: It is much easier to ask 
leading questions than to ask open-ended 
ones. Learning to ask good questions comes 
through practice, combined with an awareness 
of where to look for mistakes in the 
techniques as well as in the behavior of the 
pretesters. 


Once aware of how attitudes influence 
behavior, and how certain behavior can 
prevent good pretesting, you can take action. 
(If you have defined what the problem is, you 
are more than half way to the solution, 
according to Einstein). The process of 
pretesting is a fascinating study in people's 
way of perceiving pictures, in their way of 
thinking about Eraditions, about their 
problems and needs, about themselves in the 


world, about others... It is one of the best 
ways to learn to communicate with other 
people. 


The key is to keep the communication with 
your audience very open and frequent - on 
ideas and projects as well as on the 
communication materials designed to support 
these projects. 


When tested in Nepal, 32% of the respondents 
interpreted it as "dead person" However, 
there were also answers like wild animal, 
crow, shore of the river, aeroplane and god 
praying. Not always easy for the pretester tc 
keep a straight face, but with good training, 
most of the interviewers did a good job. 

The central idea of pretesting is to test 
the pictures, not the people. Many pretesters 
tend to forget this, and make people feel 
insecure or angry. 


The villagers must believe you when you say 
that they have something to teach you, and 
that you want to learn from them. Respect is 
the key word in this process. 


& 
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D. ANALYSING THE RESULTS OF PRETESTING 


Today, computers are frequently used to 
analyse research results. Computers cut down 
the time needed to analyse studies with a 
large number of detailed questionnaires and 
hundreds of respondents. 


However, computers have their limitations, 
and are most often not suited to analysing 
pretesting results. The main reason is that 
in a pretesting interview you usually have to 
ask open-ended questions (like "What do you 
see in this picture?"), and the answers have 
to be recorded word by word. There are 
usually so many different answers to each 
@eescion that it is very difficult to code 
them into a system that is needed for the 
computer. Also, in many instances you need to 
probe, you ask new questions based on what 


the respondent says. Therefore, each 
interview will be different, even though your 
aim is the same - i.e. finding out how your 


respondent understands the communication 
Materials, and if he/she has any suggestions 
for improvement or changes. 


It is tempting to use the new technology 
available to researchers these days, but 
computers are usually not necessary or 
Suitable for pretest analysis. 


In a normal pretest, you will interview 
20-50 people, and _ this number is not 
sufficient to make a computer analysis 
worthwhile. 

Many people let the technology decide what 
kind of interviews they are going to make, 
instead of letting the purpose of the 
information gathering decide what is the best 
way to go about the job. Thus, people will 
develop questionnaires with yes/no answers 
(like "Understands/does not understand 
picture") because they want to use computer 
to analyse the results. By doing this, you 
miss out on the purpose of the pretesting ~ 
which is to collect qualitative information 
that can be the basis for a decision on how 
to improve the communication materials. 


Analysing pretesting results usually means 


going through the notes (see section on 


recording the results in chapter "How to 
pretest") from the inteviews, and making 
categories for the main trends. An example 


from a flipchart on how to make rehydration 


solution: How many people could interpret 
picture no. 1 (2,3,4..4 ete For each 
picture, note specific things people made 
remarks about - positive and negative. Then, 
note suggestions people made for’ each 
picture. Then - how many people understood 


the whole message after having gone through 


the flipchart? (Here you could make a 
distinction between people who got’ the 
message through just interpreting the 


pictures, and people who got the message when 
being taught by the health workers. This 
distinction will give you an idea about how 
well {people .=pick up things ‘through just 
ISOKingG== at. pictures, i.e. if they see 
pictures as a teaching medium, or if they 
need careful explanation of each step before 
comprehending he story or the message). 


Next question: Suggestions for improvement 
of the flipchart. Then, if you have let the 
users test it, you need to ask some questions 
about them (in this case probably health 


workers): How many people understood the 
message after having been taught by the 
health worker? What are the skills of the 
health worker - what are strong points and 


weak points in his/her teaching? 


If the pretesting 
interviews have been 
recorded properly, 
you can analyse the 
results by going 
through the notes 


and make categories 
for the main trends 
for each picture. 


The final question, which can be answered 
by a combination of suggestions from the 
health workers, their trainers (if they are 
present) and yourself, is - what do the 
health workers lack in the way of teaching 
skilis-; i.e. to be abtle= eto. use . the 
communication materials in an effective way, 
and how can they be trained to acquire these 
skills? 

The answers to these questions should then 
be discussed with the appropriate people, and 
the necessary action taken to change the 
materials, and/or to get people started on 
planning training workshops for the health 
workers. 

Not all pretesting exercises will be as 
elaborate as this example discusses - in many 
instances it will be a question of testing a 
Single poster, and making suggestions for how 
it can be changed. However, you always have 
to bear in mind the skills and the practical 
Situation of the user of your materials - it 
is no use coming up with good materials if 


people do not know how to use them to educate 
the audience. 
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E. WHERE AND WITH WHOM TO PRETEST 


Communication materials and ideas should 
usually be tested with several different 
people, depending on the target audience and 
users. 


For instance, a flipchart describing why 
and how to make rehydration solution for 
children with diarrhoea, should first be 
tested with mothers, fathers and grandmothers 
in the area where it is to be used. The 
flipchart should be tested for acceptability 
of the idea (which ideally should have been Jf you take the "easy way out" and test with 
researched before making the materials), and People in your office, your results will not 
for comprehension of the individual pictures °° worth much. 
as well as the whole message. Then, the 
flipchart should be tested and discussed with 
official and unofficial leaders in the 
community, to get their suggestions and 
enlist their cooperation and support for the 
campaign. Finally, the flipchart should be 
tested with the users, in this case the 
heatih workers who will teach the mothers, 
using this flipchart. After testing it with 
the users, the health workers should use the 
flipchart on the target audience. The 
pretester will then observe their teaching 
techniques and ability to understand and use 
instructions. This test should ideally be 
done in cooperation with the people who are 
going to train the users in health education 
techniques. (see chapter "How to pretest?" 
for examples of how such testing can be 
done). 


Materials should be tested with the target 
group for whom they are intended, or with 
people close to the target group. For 
instance, a poster made to promote 
breast-feeding in rural areas’ should be 
tested in rural areas with the mothers who 
live there. If the poster is tested jn an 
urban area only, there is a possibility that 
the rural women will not relate to it. 
Similarly, if you test the material with your 
colleagues only, you have a very little 
chance of finding out what the real problems 
of the materials are. Discussion with 
colleagues is, of course, a good place to 
start the process, but if that is also where yo 
it stops, it is not of much use. there. 


If the target audience is school children, 
u should go to a school to test your visual 


Sometimes it is very tempting to test the 
materials with people who pick up the ideas 
quickly, and can give you good suggestions on 
how to improve the materials. However, you 
should bear in mind that even if this group 
is important, especially because they will 
normally give many suggestions (though keep 
in mind that if they are very well educated 
and different in many ways from the farmers 
and the mothers you try to communicate with, 
their suggestions may not work - so —- pretest 
them as well). Always test your materials 
with the weakest link of the chain, i.e. 
those with the least education. The poorest 
sector of your audience, the ones with little 
or no education, limited exposure to the 
outside world (i.e. outside their village), 
the ones trying to make a living from 
subsistence farming - they are the ones who 
may most need assistance from your project. 
If they do not understand your communication 
materials, your field workers will be in 
trouble, and the audience will not learn 
anything. It is much more difficult to test 
with this group, and it takes much more time, 
but finding out what this group - can 
understand and relate to is the most 
important part of your work. If they can 
understand the materials, the more educated 
ones can also understand them. 


A group situation can 
sometimes be difficult to 
manage - there may be 
some who are more 
interested in making fun 
of the pictures and of 
each other, than in being 
really constructive. If 
you have people like that 
in a group, the best way 
to deal with them is to 
try to get the interest 
of the rest of the group 
without ignoring or 
putting down the one(s) 
trying to be funny. 


done with individuals 
and/or with groups, depending on _ the 
situation and on the materials, and on the 
purpose of the test. If you test with groups, 
you will find that people respond more 
easily, and that there is always somebody in 
the group who will understand and give 
suggestions. If the group is not homogenous, 
i.e. if there are illiterate people as well 
as people with high education, you may find 
that those with little or no formal education 
will not say anything for fear of appearing 
"stupid" in. front of those _wlea higher 
status. This does not mean you cannot test 
with groups - it just means that in many 
cases you need to do both. When you test with 
groups, many people will feel freer to 
discuss the issue and/or the picture being 
tested, and you may find out what they are 
concerned about in this connection, e.g. 
regarding local beliefs and attitudes. Also, 
people will often come up with suggestions 
for improvement when you test in a group, and 
you have the advantage of both getting the 
suggestions and of hearing the other group 
members' reactions to these suggestions. 


& 


Testing icaumeve 
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F. WHEN SHOULD YOU PRETEST ? 


Pretesting should be done as early as 
possible in the production of communication 
materials. The first test should ideally be 
done when the materials are in a very rough 
stage, €-g. pencil sketches of ideas. The 
reason for testing them at this Stage is that 
as neither much time nor energy has been 
invested in the materials, it may be easier 
to make changes. People you test with may 
also be more interested and willing to make 
suggestions and give critical comments if 
they see that the materials are in the draft 


stage. People are less likely to criticize 
something that is already printed, or 
something that looks very elaborate and 
fancy. 


Testing at an early stage can be done ina 
location fairly close to where you work. For 
instance, if your materials are made for 
farmers in the mountains, you could test them 
with farmers in the hills to get a general 
idea of the reactions and _ suggestions. 
However, at some stage in the testing, you 
should attempt to reach your real audience. 

If you test at an early stage, you will 
have time to make the necessary changes, and 
test the materials again before they are 
produced. In some instances, especially when 
you start pretesting, you may have to go out 
five or six times before you are satisfied 
that your audience understands the materials. 
However, once you get more experience in 


testing, you will get a better feel for what 
people 


different kinds of like and 


Pretesting at the stage where the poster is 
already printed, is of little use. 


Materials should be tested at this stage, 
when they are just rough drawings. 


ET 


understand, and the time needed for 
pretesting will be less. 
If testing is done at a late stage, just 


before the materials are going to be printed, 
you may have trouble with changing what is 
needed, and doing a new test. At this stage, 
there is often pressure from superiors to go 
ahead and produce, because the materials look 
"ready". 


In other words - start your testing as 
early as you can, and continue testing at 
different stages of the process - each time 
you have made changes in the materials based 
on suggestions from the field. If you follow 
this procedure, you will learn more, and your 
superiors will not force you to produce 
before you are ready. 
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G. HOW MANY PEOPLE SHOULD YOU TEST WITH ? 


This depends on the answers you get in 


your prestesting. The rule is - if you get 
consistent answers (ie. most people 
understand your materials well, or most 


people have problems with some parts on it), 
it is enough to ask 20-30 people. Continue 
testing until you find a trend - i.e. until 
you get a sense from enough people that you 
have found out what the problems are with 


as 


This picture (intended to symbolise danger) 
was tested in Nepal. Only one percent said it 
indicated danger. However, 61% gave responses 
that indicated the picture gave them negative 


feelings. The conclusion from the test: 
People do not interpret the meaning of 
pictures, or of symbols, but as long as the 


picture arouses the appropriate feeling from 
a majority of the respondents, we can decide 
that this is a good basis from which to start 
teaching. We could only reach this conclusion 
by testing with enough people to feel sure 
that the negative feelings were consistent. 
People gave answers like "something 
frightening", "monster", 
"ghost", "wild person", etc. 
(From with Pictures in 
Nepal"). 


"skull and bones", 
"dead person", 
“Communicating 


your materials. If 20 people interpret you 
baby-scales as a clock, you know you wil 
have to re-design the picture. 

If the answers differ a lot, if you ge 
many different reactions with no _ obviou 
trend or pattern, you may have to test wit 
up to 50 people. The rule is the same - tes 
until you feel fairly sure you know th 
problem. 


ine 2this meant./to indicate ‘thar 


picture, 
breastfeeding is better than bottle-feeding, 


82% of the respondents 
chagiaren™.. 19%. “said 
breast-feeding", “buted as% gotsemne 
"message", based on the tick and cross 
indicating good and bad. Conclusion: Abstract 
symbols that do not elicit any emotions are 
of very limited use, unless they are taught 
to the audience. (From "Communicating with 
Pictures in Nepal"). 


saw "Adults 
"bottle-feeding 


and 
and 


Once you have definea the problem you have 
to ask yourself and also some of the 
respondents - WHY is there this problem with 
the materials? When you have defined these 
reasons, you will have a basis to decide what 
the next version should look like. For 
suggestions to this, group testing is good. 
People in a group will often start discussing 
how a picture could be improved, while it may 
be more difficult to get individuals to come 
up with suggestions (see also "Where and with 
whom to test"). 


When you have a good idea of what you need 
to change, and what your next version is 
going to look like, you have to go back to 
the drawing board. When your next version is 


ready, this has to be tested, too. There 
might still be things that need to be 
changed, and when you make new versions, new 


problems crop up. 


How long do you have to go on testing? 
When can you decide that your materials are 
good enough to produce? This is a difficult 
guestion, and there are no definite answers. 
Some guidelines: If you are testing single 
pictures, you should go on testing and 
changing until at least 50%, preferably 70%, 
understand the idea of the ptcture. If you 
are testing pictures in a series, the same 
rule should apply for every picture more 
than half of the respondents should be able 
to interpret the picture correctly. Regarding 
the story or message that is meant to be 
communicated by the flipchart, understanding 
will depend to a large degree on previous 
experience with pictures in a series. Many 
people who are not literate and not used to 
being taught with pictures, may not see any 
connection between these pictures at all, 
although they may be able to interpret each 
picture correctly. If this is the case with a 
majority of your respondents, there isnot 
much you can do with the visual itself 
except making sure that the individual 
pictures are being understood. You may have 
to accept that only 10-20% of your 
respondents get the whole message through 
looking at the flipchart. 


In this case, point out to the trainers of 
the field workers, that they make the field 
workers aware that the audience is not used 
to being taught with these materials, and 
suggest training for the field staff. It is 
also advisable to print a message on the 
flipchart indicating that the pretesting 
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a 


Group testing can be good, especially if you 
are testing several versions of e.g. a 
poster. Here - testing a poster about "Taking 
care of your school-books" in a _ primary 
school in Nepal. 


showed up difficulties with pictures in a 
series. Give the field workers some advice on 
how to deal with. Printing a little "warning" 
to flag a difficult picture can also be done 
in other cases where you find that a concept 
is very difficult to get across, and that 
even after trying out five or six different 
versions immediate understanding is low. Such 
a warning could read e.g. "Pretesting has 


shown that this picture is difiiicuit tor 
people to understand. Please explain 
carefully, and then discuss the information 
with the audience, to make sure they have 
understood" This method should not, however, 
be used aS a substitute for proper 
pretesting. 


Sometimes you may be pressed to produce 
before you are satisfied with the way the 
audience understands your communication 
materials. In such cases, you should take 
care to get good feedback from:the field - 
preferably you should also visit the field 
and observe the materials being used, and 
people's response to them. Such feedback 
from field workers and through your own or 
colleagues’ observation - can be used to build 
up a “case” to convince the decision-makers, 
or whoever pressed you to produce before you 
were ready, that the problems encountered in 
the field could have been avoided if you had 
had more time for pretesting. 
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H. WHO CAN PRETEST ? 


In principle, anybody who is trained can 
carry out a pretest. There are, however, 
pro's and con's for the different categories 
of people who normally would do this job. 
Here, we will present these pro's and con's, 
and then some examples of how pretesting has 
been done in government departments and 
international organizations. People who will 
want to carry our pretesting can then use 
these arguments and discuss them in their own 
local and organizational context, and then 


decide on who should be trained to do this 
job. 


a) Researchers. People with professional 
background in research are in many instances 


well qualified to pretest. They know (at 
least in theory) information gathering 
techniques, sampling, and information 
processing. They can take communication 
materials to the field and bring back 
quantified information about the 


understanding of your messages. 

Some arguments against using researchers: 
They may be academics and removed from the 
reality of field problems. Researchers may 
also not understand the purpose of pretesting 
properly - and may want to make it a "proper 
academic exercise", meaning that they will 


want to interview at least 100 people, and 
Will mnotwsaccept that conclusions.) about 
changes are made unless they are 


statistically sound. This will prolong the 
time needed to do the pretesting especially 
if it has to be done more than once (which 
most pretesting ©f communication materials 
should be). Some researchers may be rigid 
about "living up to their professional 


standards", others may agree to carry out 
pretesting based on the purpose and 
principles suggested by the project people - 
or in cooperation with them. (The suggestions 
in this manual are to facilitate the process 
of improving communication materials within 
the practical time- and money-limits faced by 
most development projects, and not to conduct 
statistical research). 

Researchers may not have been involved in 
the planning of the materials, and the 
planners and/or. artist who have 
planned/developed the materials may not be 
willing to disten to the criftiezems orougnt 
back by the researchers.. The researchers' 
non-involvement in the development of the 


materials can also be an asset - they have 
nothing invested in the materials, therefore 
they can criticize very freely - but the 


implementers will often be defensive about 
such CYltCicisn. 


Researchers can become effective 
pretesters through close cooperation with 
project planners and implementers as there 
has to be agreement on the purpose of the 
testing. 

The researchers' lack of knowledge about 
the content of the materials may also be a 
drawback. In many cases, they may not be 
familiar with the context in which people 
Operate or make decisions (e.g. in nutrition 
or food-habits), and may not be able to probe 
in the right direction when they get a lead 
from a respondent. In this case, it is also 
crucial for the implementers and planners to 


brief the researchers thoroughly before the 
testing is done. 


b) Planners and implementers. The advantage 
of having this group (or individuals withir 
it) pretest the materials, is that they knov 
the project well, they know the purpose of 
the communication materials, and they knov 
the subject. Therefore, they can interpret 
feedback and suggestions within a_ wide 
context than people without such knowledge of 
the content are able to. They are also more 
likely to use the feedback and suggestions 
from the target audience through pretesting, 
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if they have decided to do as good a job as 
possible within the limits the project sets 


for them. Pretesting by this group will 
create a healthy atmosphere within the 
project of willingness to learn and to 


establish closer links with the audience. 
Having planned, but not actually produced the 
drawings/photographs for the materials 
themselves, they will have enough distance 
from the artistic side of the materials to be 
able to criticize freely. 


However, their close connection with the 
project may prove to be a disadvantage in 
some cases, if the people who are pretesting 
cannot manage to get enough distance from the 
project to be able to look at the task at 
hand, and look at it from the audience's 
point of view. In some cases, planners and 
implementers may be extremely defensive about 
their project and about their communication 
materials, and may go to the field to have 
their opinions confirmed or their projects 
sanctioned rather than to raise questions 
about it. If the planners and implementers 
are not experienced in the techniques and 
practices of pretesting, they may not be 
aware that they are actually doing this, and 
may come back to the project office with a 
lot of positive answers to their "guestions". 


As time frames for projects are most often 
underestimated, to meet target dates, 
pretesting will often be eliminated being 
most often thought of as a non-essential 
practice by those who evaluate the project 
(even though this is slowly changing in many 
places now - more and more people see the 
necessity of thorough pretesting). 


When feedback and suggestions from the 
field are brought to the artists who have 
developed the materials, there is possibility 
for a communication gap - the artists may not 
want to listen to the suggestions, because 
"the villagers do not know about art,and this 
is done correctly", or other arguments along 
the same line. If the planners’ and 
implementers do not explain to the artist the 
purpose of the communication materials as 
well as the purpose of pretesting, the result 
of the revised materials may not be what this 
group had intended. Artists are often treated 
as mere "doers", they are told what to do, 
and not given any reasons for why changes 
have to be made. This produces resentment. 
With more sensitive handling they could 
become better educational artists. 


Artists can be good pretesters - if they have 
an interest in educational materials. 


c) Artists and/or photographers. The 
advantage of this group carrying out 


pretesting is that they may learn directly 
from the audience how their materials are 
perceived. If the artists are really 
interested in developing skills to make 
better educational materials, it is essential 
that they go to the field and experience 
people's reactions and suggestions to their 
drawings or photographs. After going to the 
field several times, each time testing a 
version based on suggestions made by the 
audience, the artist will slowly acquire a 
feeling for the kind of materials people 
understand and like. This will, in time, cut 
down the time and resources having to be 
spent on pretesting, because the artist will 
presumably be closer to "right" each time. 
However, one should be careful not to use the 
artist's acquisition of skills in this area 
as an excuse for forgoing pretesting. Even 
with a well-trained artist with skills and 
interest in educational materials production, 
pretesting is always necessary. 


. ‘ . eS < 
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Artists who work within a project, or a 


series of projects, will be very useful 
resource persons for people who work in 
related areas as well, e.g. an artist working 
in a nutrition project, and has been field 
testing his materials several times, will be 
able to give advice to other artists and/or 
planners/implementers working on a project 
next door. The accumulation of such 
experience can ultimately be used to form 
guidelines for a "visual language" that is 
most easily understood by the different 
target groups. Working towards such a goal 


might also inspire the artists to do a better 
job. 


One reason against using artists as 
pretesters is that they will be defensive 
about their work. Very few artists have an 
educator's outlook necessary to become a 
really good educational artist. 


Many artists will be reluctant to listen 
to villagers' opinion of their products. Some 
will feel that it is beneath them to ask 


uneducated people whether oor not they 
understand the intended message, and will 
hold the opinion that if people don't 


understand, it is the people's fault and not 
the artist's fault. This attitude may be very 


Field workers or trainers can be goo 


“pretesters because of their extensive fiel 
experience. 


widespread among artists who primarily view 
their work as ART rather than as 
communication tools for education. 

Most artists may not see the importance of 
your project aims or. that of the 
communication materials you want to produce. 
The idea of establishing a close 
communication and co-operation with villagers 
May not appeal to them. 


Those artists who become interested in 
learning more about how people understand 
pictures, and want to become better at 
communicating with villagers, will gradually 
develop the skill and the ability to test 
their own materials as their curlosity 
overcomes their defensiveness. They will get 
less attached to their product and more 


attached to the task of communicating. 


Artists who are attached to a project can 
be gradually trained (but even they have to 
have the interest and the attitude described 
above) to become better educational artists. 


If the artist is hired from an outside 


agency 
which deals with commercial art, there is 
less of a chance that he/she will be 
interested in and capable of doing the job 
your project needs. This is a strong argument 
for getting an artist attached to your 


project or to a series of projects - at least 
on a part-term basis, and then training him 
or her to do the kind of work needed. 


d) Field-workers and villagers. The advantage 
of using e.g. health workers to do your 
testing, is that they are familiar with the 
audience that you want to pretest your 
materials with. They have (at least in some 
cases) good credibility in the community, and 
many may come from this community. The same 
can be said for village leaders, or other 
influential people in the village, whom the 
pretester might ask assistance from in his or 
her work. 


However, experience has shown that in most 
Peses it is not a good idea to use field 
personnel to carry out pretesting. Despite 
training in testing methods, and orientation 
about the objectives of testing, field 
personnel and: influential villagers will 
often test the audience rather than the 
Materials. They will be interested to show 
that their villagers are "intelligent" and 
Can understand the materials, and will ask 


leading questions like "Do you understand 
mas picture", and "Do you like this 
picture?" The field workers are often also 


very much aware of the status disparity 
between them and the villagers, and may be 
interested to show this off to the outsiders 
i.e. the project people who have come to 
pretest. Thus, they may not behave in a way 
that is conducive to obtaining people's ideas 
and suggestions about communication 
Materials. 


Having discussed advantages and 
disadvantages of all these groups, who would 
then be the best people to carry out 
pretesting? In each situation it will depend 
On who is available and interested to work at 
this, as well as whether or not they are ina 
"Suitable" position. However, each project 
that wants to start pretesting activities 
Should look at and discuss the arguments in 
this chapter thoroughly, and then decide what 
fits their situation. 
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sometimes 
Materials 
villagers' reaction to the materials. 


leave 
places, 


people will 
in public 


communication 
and observe 


that have worked in 
other places: A pretesting team can be 
established, consisting of an artist and a 
planner/implementer, occasionally joined by 
their supervisor. The artist (or ina 
bigger organization - several artists) can be 
gradually trained to do the job by himself. 
When the planner/implementer is comfortable 
with the artist's pretesting skills, he can 
send him out alone, and he will also take 
account of his suggestions. 


some permutations 


Another combination is. an artist, a 
planner/implementer and a researcher, if the 
organization has a research unit attached. 
They will all have different purposes in the 
beginning, and it will most often be the 
planner/implementer who has to hold the team 
together, and guide it in the direction 
he/she feels will be useful to his project. 


In summary, we could say that a 
combination of skills is necessary in a 
pretesting team, and that it is a good idea 
to be clear about your purpose with the 
testing from the very beginning - not only 
what you want to find out about your 
communication materials, but also how you 
want to build up the pretesting skills in 


your project or organization. 


as 
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HOW TO CONVINCE YOUR SUPERIORS 
THAT PRETESTING IS NECESSARY ? 


While pretesting is frequently discussed, 
it is not often practiced, and it is 
difficult to find results from other places, 
that may help convince your bosses that this 
is an essential part of .developing your 
communication materials. And - the hard facts 
show that unless your superiors are 
convinced, you will get neither time nor 
money to carry out pretesting. 


Superiors are most often convinced by 
results. If you can show them how you have 
tested communication materials, what the 


response and suggestions of the audience was, 
how you changed the materials, tested them 
again, and finally arrived at a product that 
was well understood, they will usually be 
convinced enough to let you continue with the 
practice, and will also give you time and 
resources to work with. If you can _ show 
concrete results from the field about what 
has happened as a result of villagers being 
taught with these communication materials, 
you have an even stronger case. Another idea 
is to record interviews with villagers on 
tape, where you interview them not only about 
the comprehension of the materials, but also 
aboubeewiet- they think about the idea of 
testing Materials this way. For such 
interviews, it. is often a good»idea to get an 
influential villager who supports and has 
watched some of the pretesting. 


However, you will not have any examples of 
materials until you have been to the field, 
and have had the- time and resources to 
collect them. There are a couple of ways to 
approach the issue in the beginning: You can 
show your superior some of the examples used 
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in this manual, and discuss the arguments 1 
"why pretesting" with him or her. “"ia 
argument that pretesting saves a lot of mone 
in the long run is often an important one a 
this stage. You may then be able to convinc 
him to give you time and resources LOrGe 
experimental pretesting within your project 
If you choose this direction, it is a goo 
idea to have a concrete suggestion for wha 
you want to do - a version of the visual yo 
want to test, where you want to test it, wit 
what kind of audience, how much time it i 
going to take and what it will cost. 


Another approach is to spend your ow 
time, whenever you are in the field for othe 
purposes, and slowly collect responses t 
materials you are working on. This is a mor 
direct approach which will produce = som 
results (different stages of a picture, fc 
instance) that can be used to convince you 
superiors of the necessity for pretesting 


Yet another approachiis @to-use example 
from other projects where people ar 
practicing pretesting, and where examples c¢ 
materials in different stages are available 
You could even ask the people who have bee 
pretesting: for -thiss projece; to meet yol 
superior together with you. If you choos 
this approach, it is also advisable to have 
concrete plan to suggest to your superior 
some materials you want to test, how you war 
to do at, what it wild "oe eccc. 
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of a poster in four 
different stages, intended to convey the idea 


This is an example 
of a "healthy baby". People liked the child 
in stage I, they thought he was healthy and 
happy, but he had oranges close to his feet, 
which conveyed the idea that "you have to 
give your child oranges to become healthy". 
Poor people cannot afford oranges. Second 
Stage: People said this was a city baby 


(which it indeed was), because he was wearing 


diapers. So - back to the drawing board. The 
third stage: This baby is too fat, and 
besides, he could be that way just by 
drinking mother's milk. He is too young. 


Went back to the child that 
people liked so much (first stage), and got 
his picture without oranges. Villagers, 
health workers and teachers agreed that this 
picture really conveyed the idea of a 
healthy, happy child. 


Fourth stage: 


An example like this can be a good way to 
demonstrate to people the need for and the 
usefulness of pretesting educational 
materials. 

It is not possible to say in general which 
approach would work best with your superiors. 
from dealing with them 


You should know, 
before, which method is most likely to 
succeed. 


Once you have convinced your superior that 
pretesting is essential and should be used in 
your project, you will go ahead and do the 
work. When you are comfortable with the 
techniques and practices of pretesting, it 
may be a good idea to invite your superior on 
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a field trip where you demonstrate to him or 
her how pretesting is done, and what people 
think about it. It may not be possible to get 
your superior on a separate field trip to 
observe pretesting, but it may be a good idea 
to include pretesting demonstrations in a 
field trip that is carried out primarily for 
other purposes - as long as you are in an 
area where the audience BSB your 
communication materials can be found. Once 
your superior has observed pretesting, he or 
she will often be fully convinced of the 
usefulness and necessity of this practice, 
and will give you a freer hand and more 
resources to carry out pretesting. 
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NOTE ON TRAINING PEOPLE HOW TO USE 
COMMUNICATION MATERIALS 


Communication materials very seldom work 
on their own. Most materials are designed to 
be used by a teacher or extension worker of 
some kind. The design of the materials also 
depend on the skills of the people who are 
going to. make use of the materials. Very 
often, this concern is forgotten, and 
sophisticated materials are developed for 
extension workers who know very little about 
using communication materials in their work. 


Therefore, it is important to assess the 
skills of your users, as well as the 
perception of the audience to your materials. 
If you are a project planner/implementer, or 
Becesearcher, or an artist, it may be an idea 
to initiate cooperation with the people and 
the institutions training the extension 
workers for whom you design the materials. If 
these trainers can come with you to the 
field, you can assess the skills of the 
workers together. Get the trainer to teach 
you what to look for, so that you will be 
able to assess these skills later. However, 
it is advisable to get the trainer to come 
with you a few times in the beginning, until 
you are sure about what he is looking for. 
Later, you should also get him or her out to 
the field with you from time to time, if he 
or she is interested. By doing this, you will 
Take sure that the trainers are really aware 
of what the problems are in the field, and 
that their training programmes will be 
designed to deal with these problems. 


Many trainers go to the field very seldom, 
and they may not even be aware that many 
illiterate villagers have problems 
interpreting the communication materials that 
are produced (this has been shown in several 


countries). Indeed, they may not be aware 
that their own students - the extension 
workers - in many instances do not understand 


the communication materials. By pretesting, 
you will hopefully improve on this situation, 
but you would be wise to bear in mind that 
most people you work with, including the 


trainers and the extension workers, will 
assume that people in villages understand 


Pictures of any kind, because they have not 
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still 
to the villagers, 
rather than discuss the problems with them. 


Many field-workers 
"delivering the message" 


asked the villagers if they understand the 
pictures. If they are using the lecture 
method, they may never find out that people 
do not understand the pictures. Instead, the 
workers may blame the villagers for not doing 


what they are taught to do by the workers 
teaching them with pictures they have not 
understood. Any so the communication gaps 


widen.... 
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Initially, you may find that people you 
work with - including bosses, trainers, 
artists, etc. - will not believe that people 


have trouble with the communication materials 
you are testing. Many people who grow up ina 
visual environment, like a town, will not 
even think about the possibility of somebody 
not being able to interpret pictures, or 
interpreting them in a different way from the 
"obvious" intention. Use a tape-recorder to 
convince those people about the reality of 
the field situation. It is even better to get 
them to come out to the field with you. 


However, when you take trainers to the 
field, you may have to "train" them in 
advance. Teachers often have an irresistable 


urge to jump into a situation and take it 
over, and teach the extension worker how to 
do his job better. Discuss the purpose of the 
exercise with the trainer before going to the 
field, and try to get him to agree to just 
observe and not interfere in the situation. 
After the extension worker has conducted his 
interview, you could all three sit down and 
discuss what kind of skills are needed - 
which would also make you and the trainer 
aware of how the extension worker looks at 
his own teaching abilities, and what he 
thinks he needs. Such a discussion is an 
interesting and useful exercise for all the 
participants. If you have several extension 
workers trying out your communication 
materials, one by one, you can have the 
discussion in a larger group. This is often 
better than on a one-to-one basis, and the 


extension workers might be more willing to 
discuss their training needs and experiences 
if they can support each other. 


Based on the results of you) 
field-discussions, and on the results of the 
testing of your visuals, the trainer and hi: 
colleagues can plan the necessary workshop: 
and/or training courses that are needed t 
give the field workers for the project the 
necessary teaching skills. Depending on youl 
position in the project, and on your interest 
in the training, you could be involved ir 
parts of such training courses. Examples o! 
the different stages of pictures or materials 
will be really useful in such training, tc 


help the trainees understand what people 
understand, and why, and how you find this 
out. (The basic ideas in pretesting are ver 


closely related to the basic ideas of gooc 
teaching). By taking part in such workshops 
or courses, where the trainers are using the 
materials you have designed,, you will get = 
better idea of what kind of materials are 
suitable at this level (you may sometimes be 
asked to produce materials for use ir 
training courses), and about how muct 
additional training the field workers will 
normally have access to. 


Putting all this information together 
from the villagers, the field workers, th 
discussions with trainers and field workers 
and the training courses - you will graduall 
get a much better idea of what kind o 
communication materials should be develope 
for your projects. 


Working with trainers in the field can be | 
very useful experience. 
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FEEDBACK FROM USERS OF 
COMMUNICATION MATERIALS 


Field workers who 
use the materials 
are your best 
sources for 
feedback about the 
usefulness of these 
as educational 
tools. 


Even if you pretest your materials, 
evaluate their effectiveness once they are 
distributed to all the users in the field. 

There are several ways to get _ such 
feedback. One way is to go to the field 
yourself - or send your colleagues - to 
observe how the materials are used, and talk 
to the users as well as the audience about 
it. This would be very similar to what you do 
when you pretest the materials with the 
users, except if you go to the field when the 
Materials have been in use for a while, you 
might find out more about how effective they 
are. After teaching with the materials for a 
while, the users will probably feel more 
comfortable about suggesting changes in the 
materials, because they will have experienced 
what the problems are by using them. 


But - you cannot visit everybody, and 
there are several ways to get feedback 
directly from the field workers as well. You 
can send out a letter with a questionnaire, 
asking them to give their ideas and 
Suggestions for improvement. However, letters 
often tend to get lost or forgotten, and it 


may be a better idea to print e.g. an extra 
page on the flipchart, asking them the 
questions you think are appropriate. You can 
instruct them to tear off this page and send 
it to your project, once the field worker has 
used the flipchart for a couple of months. 
This way, he will always be reminded (each 
time he uses the flipchart) that he can write 
down suggestions for improvement, based on 
the experience he gets and the effect his 
teaching has on the community. 

Many field workers will respond positively 
to a request about feedback - it shows that 
the project authorities (which to most of 
them may be a very abstract body) are 
interested in their opinion and suggestions. 
However, if you do ask for feedback, you also 
have to explain what you are going to use 
this information for. If you do not have a 
budget for revision and reprinting of this 
material, do not promise that it will be 
revised. However, the information from the 
field can be used to improve the next set of 
materials you make. This should be clear to 
the field workers, so that they know what to 
expect from the work they put into the 
evaluation. 
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SOME QUESTIONS FOR DISCUSSION 
~AND SOME THOUGHTS ON THEM 


a) Is a bad picture better than no picture? 


There is a belief among some people that 
even if a picture is bad, it will be better 
than nottevig a picture at all, as long as 
the teacher/field worker explains it. One 
can argue for and against this belief or 
statement. The argument in favour of this is 
that people remember an image once they have 
understood it well - they remember it better 
than they do remember words only. And chances 
are that the person who produces the bad 
picture may be interested in the idea of 
teaching with pictures. He may learn 
gradually to improve his pictures, and start 
making really good ones. This presupposes, 
though, that the person who is making and/or 
using the picture is open to the fact that it 
might be bad (i.e. not understood), and is 
willing to work to imporve it. 


The argument againstthis belief is - why 
teach with a bad picture if you can teach 
with a good one? If the picture is pretested, 
it can be improved so that people will 
understand it well. The more confusing a 
picture is, the more time and energy people 
will use in trying to understand what it 
means. Which part the eyes or the ears - 
will use the most concentration? If a person 
sees a picture he does not understand, or is 
confused by, and has it explained to him - 
what will remain in the brain, the confused 
picture or the explanation? There is no easy 
answer to that question. Except ty to 
develop good pictures instead of bad ones in 
the first place. 
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Another argument against the belief i 
that the less clear the picture is, th 
greater the demand on the teaching skills o 


the users, e.g. the health workers. Takin 
the reality of the situation in mos 
developing countries into consideration 
field staff in general are not equipped wit! 
good teaching skills. Thus - _ confusin 
communication materials are not what the 
need. 


b) Can a picture alone carry a message? 


There is no clearcut answer to this 
question, but in general, for the kind o: 
development purposes we are discussing ir 
this manual, the answer is usually no. 3 
poster will not make people do something by 
itself - unless it is e.g. information about 
when to bring your babies to the clinic fo: 
immunization (and the audience is _ ali 
literate), and already convinced that the 
children should be immunized. A picture alone 
will not convince people to change thei 
behavior a mother will not decide to have 
her children immunized because she sees 
poster that says 


"immunization is good for 
children". She might become interested ané 
start discussing the subject with her 
neighbour, or with some socially influential 
women to whom she goes for adivce on healtk 
matters. “ff “they say it “ws Sgoe08 ror the 
children, 


she might seriously consider it. 


This picture was used by an organization in 
Nepal to convey the idea that tuberculosis 
germs can spread from one person to another 
by breathing or coughing. When testing this 
picture, we found that nobody got the 
message, except people in one village where 
the organization has worked. Here, people 
remembered the picture and could explain what 
it meant. When people have understood a 
picture well, they will usually remember it - 


even if the picture is very crude, like this 
one. 


She might discuss it with her husband and get 
his consent. She might then go to the clinic 
and get one of her children immunized. 
However, if the nursesin the clinic do not 
explain that the child might develop some 
fever, and that this is quite normal and 
nothing to worry about, the mother might not 
return for the second or third doses. As she 
sees it - her child got fever from the 
injection. She does not see the point an 
making her child sick. A poster by itself is 
not gOing to convince her to go back again, 
but a health worker who understands the 
situation, finds out why the mother is 
against immunization, and explains to her - 
with the help of a poster or a flipchart - 
why immunization is necessary and what may 
happen if the children are not immunized, 
might convince the mother. 

Pictures or communication materials seldom 
work on their own. But, if used properly by 
someone who knows how to teach and how to 
listen, they can be very powerful tools in 


helping people to improve their lives. 
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c) If a person interprets a 
iece, and does not 
different parts in a picture, is it likely 
that he or she will correlate the 
different pictures in a story (if you are 
testing e.g. a flipchart)? 

No, not really. Visual literacy usually 
develops in stages. First, a person has to 
learn to interpret the single pictures or 
images. If there are several different parts 
in a picture - e.g. a house, a tree, some 
flowers, a family with children playing and 
some animals, the person who has not had much 
experience will look at each part separately. 
The next step is to see all these things 
together and interpret the situation. It may 
take a person several times of looking at 
different visuals before he gets to the stage 
where he is comfortable with the idea of how 
a picture "fits together" The next step is 
then to teach the person how and why pictures 
in a series are connected to each other, and 
that the person in one picture is the same 
person in the next picture. It may take a 
while before someone is familiar with this 
concept, but by frequent exposures to picture 
Stories or flipcharts, he or she will learn. 

In summary - the important thing is just 
not to assume that people interpret a picture 
Or a story in a certain way. 


icture piece by 
correlate the 


Even if people understand the images in a 
picture, they may interpret each part 
Separately. When this picture was tested in 
Nepal, some villagers said they could see a 
house with a door and a window, and then 
there were some lakes (the artistic “blobs7s 
which had no purpose). The tree was 
interpreted as a flower. Thus, they saw no 
correlation between the different parts of 
the picture regarding the scale. This is not 
an uncommon phenomenon. 


it was 


When testing pictures in a series, 
found in Nepal that people do not necessarily 
read the series from left to right (less than 


50% started from the right), and few see any 
connection between the pictures. These are 
concepts that have to be taught to people who 
are not experienced in "reading" pictures - 
which is true for a majority of villagers in 
most countries in th third world. (Examples 
from "Communicating with Pictures in Nepal"). 


d) What is best to use - photos, drawing or 


cartoons? 
The answer to this question is - it 
depends on what kind of communication 


materials you are making, the quality of the 
printing facilities available, and the kind 
of pictures your audience is used to seeing. 

Studies done in various countries show 
that in general, a photo without background 
or a shaded line drawing are the styles most 
easily understood by illiterate people, or 
people who are not used to reading words or 
pictures. The question of whether to use a 
photo or a drawing depends on your budget 
(photos are more expensive to print), the 
subject matter (if you are showing e.g. 
malnutrition in different stages, photos are 
better), and on the quality of printing 
available to you. Photos are more difficult 
to print than drawings, and it is often 
better (i.e. people will interpret more 
easily) to have a shaded line-drawing that 
comes out clearly, than a photo that is 
poorly printed. 


This style has been 


(shaded line-drawing) 
shown to be easily recognizable by illiterate 
villagers. These pictures were recognized (in 


Nepal) by 94% (water-jar), 88% (woman and 


child), and 69% (buffalo) 
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Photos can sometimes form good basis for 
drawing. IE Youreeeartist is not.eve 
experienced, he can even trace the figur 
from the photo. Drawings can be chang 
easily (e.g. if the baby is serious on t 
photo, you can make him smile on the drawin 
if needed), and it is a more versatile sty. 
than a photo. 


Cartoons can also be useful, but severe 
considerations need to be made befor 
deciding to use this style. Cartoons ar 


"unreal", and people who are not used t 
reading comic-books, will not immediatel 
understand the cartoons. They will nc 


associate the figures with real people, c 
with people from their village. For 

teaching poster e.g. for nutrition, meant fo 
people with low visual literacy, cartoons ma 
not. be 


appropriate. However, if. .syvou 
information is for urban youth who ar 
literate, and used to reading comic books 


cartoons may be just the right style. 


d) What can be done to advocate pretestin: 
more effectively? 

Examples of what has been done, found out 
and the effect of the tested materials o1 
people, are usually good tools to convince 
people about the usefulness of pretesting. Ii 
you are convinced about the virtues of 
pretesting yourself, you are the best 
advocate, together with your examples, t 
inspire people to try testing themselves. 
(See also chapter on "How to convince your 
superiors"). 


If you test your 
second version of 


the picture with 
the same group as 
you tested the 
first version with, 
you should = also 


check ‘with another 
group that has not 


seen the faxvee 
version. 
e) Should we use the same _ group. of 


respondents for  pretesting during the 


different stages of development of the 
same visual? 
Use the same group - but if so, you are 


testing for different things with each new 
version. The audience will know the content 
of the visuals, and be able to give you good 
comments on how they like the improvements - 
based on their suggestions - you have made, 
and whether your next version is closer to 
what they think the materials should look 
like. If you go back, show this community 
that you take their advice seriously, and 
they may become really interested in your 
project, and be willing and able to give you 
more feedback - both on the visuals (e.g. how 
they are being used) and on the ideas of the 
project itself. Going back to a community 
with materials that have been changed based 
on their suggestions, is a very good way to 


build credibility and rapport with the 
community. 
However, you should also test your new 


version in a community that has not seen the 
Materials before, because you need to find 
out whether they are understandable and 
acceptable to a "fresh" audience. The 
Materials, when they are sent out to the 
districts, will be used mostly with people 
who are not familiar with them, and you 
therefore need to make sure that they are as 
Close as possible to being understood at 
first sight. 


f) 


When choosing respondents, we find people 
with wide differences in age, economy, 
education, social and cultural background. 
How can we decide which of their responses 
should be used to improve our visuals? 
This decision depends on what is the 
purpose of your visual, and for whom it is 
meant (who is the audience). Your guideline 
here should be that you have to make your 
communication materials comprehensible to the 
"weakest link" in your audience i.e. the 
people with the least experience in 
interpreting visuals. If they understand the 
materials, people with more education will 
also understand. There is a danger that 
people with more education will feel that the 
materials you come up with are "too simple" 
for them, that they are not sophisticated 
enough. People who give you comments like 
that are normally not the ones who need 
health education the most, and their opinion 
and suggestions should be viewed from that 
angle. (See also chapter "Where and with whom 
to pretest?"). 


as 
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AREAS OF CONCERN WHEN DEVELOPING 
COMMUNICATION MATERIALS 


Experience in several countries shows some 
areas of comprehension that people with low 
visual literacy tend to have problems with. 
Planners and artists developing materials 
should be aware of these, as should 
pretesters. The following list is a _ short 
summary of some areas of concern to be 
checked out during pretesting. When 
sufficient experience has been gathered in a 
country, one can use this information 
(presumably from several different projects 
on several different subject areas) to 
develop guidelines for preparation of 
educational materials. However, cis is 
important to stress such 
guidelines, the be 
pretested. 


that even with 
need 


materials £O 


a) Illiterate villagers are often not used to 


the idea of a picture teaching them 
something. When they interpret a picture, 


they will often see it as a representation of 
an object (or several objects). 


b) Signs 
understood. 


and symbols are not usually 
They have to be explained. 


c) Pictures or drawings are more likely to be 
successful (in the sense that people can 
identify with the picture, and then take the 
message seriously) if the buildings, clothes, 
animals and environment are as similar as 
possible to those of your target audience. 


ad) Unnecessary details confuse people. If a 
object does not have a purpose in a picture 
take it out. Keep your pictures as simple a: 
possible. 


e) People who are not used to "reading' 
pictures, will interpret the pictures ver 
literally. Abstract ideas are not easily; 
visualized or understood. Sophisticatec 


objects that are not often found in a village 
(like a magnifying glass or a microscope) 
will also confuse people. 


f) Even if people can identify the different 
objects in a picture (like a house, a man, =< 
buffalo, children, a tree, etc.), they may 
not be able to put together the parts anc 
interpret the situation as a whole. Cause anc 
effect is especially difficult to visualize. 


g) The problem described in f) also applies 
to pictures in a series even if people 
identify the individual pictures, they may 
not see any connection between the different 
pictures in the series. 


h) Illiterate villagers do not necessarily 
read a series of pictures from left to right. 


aS 
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i) If a picture ot a person is cut off from 
the chest or the waist, it does not seem to 
pother people (example from Nepal). However, 
if you show some small parts of the body - 
e.g. hands or feet, cut off from the rest, 
people arelikely to be confused. An Indian 
study found that if you show the part, with a 
full body by the side and an arrow to the 
"cut-off"-part, people understand better. 


j) Shaded line-drawings and block-out photos 
(photos without background) are found to be 
the most effective styles in several 
different countries. 


k) People with limited visual literacy will 
have problems remembering several new ideas, 
if they are presented at the same time. It is 
better to present only one or two new ideas 
at a time. 


= 


1) Perspective causes problems for illiterate 
people. They tend to see objects ina "flat" 
Space. This means that if you draw a cat in 
the foreground, and an elephant far away in 
the background, people may interpret them as 
being the same kind of animal because they 
are the same size. 


m) The scale of objects in the pictures 
should be as close as possible to the natural 
one. This produces difficulties when you want 
to draw e.g. insects, however, if you explain 
to the audience that these insects are drawn 
this size just so that everybody can see 
them, but in real life they are much smaller, 
you may not have this problem. However, you 
should take care to draw everything in the 
picture in scale to each other. 


Nn) The positive approach often works better 
than the negative approach. This means - if 
you show a very sick child, and the reason 
for him getting sick, you should also show 
how he can get better, and how he can avoid 
getting sick. Pictures of negative things or 
sick people by themselves tend to turn people 
OLts rather than motivate them to do 
something about the problem. 


©) According to research done in Nepal 
(Dr. Linda Stone), most villagers are 
knowledgeable about their religious 
mythology, but they will generally not (as 
priests often will) connect the myths to an 
"explanation" of religious acts. 


p) If colours are used realistically, they 
can be effective and attract people's 
attention. But unrealistic colours - e€.g. 


drawings of people's faces on red background 
- may make people disregard the whole 
picture. The significance of different 
colours should be tested out in each country. 


C) 


SUGGESTIONS FOR 
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A TRAINING COURSE 


IN PRETESTING 


Some of you who have learnt about 
pretesting "the hard way" - i.e. by 
experimenting in the field, through trial and 
error, and increasingly convinced of the 
necessity and usefulness of pretesting in all 
development projects - may want fo .run. a 
workshop for colleagues and/or trainees who 
will be joining "the pretesting mafia". Here 
are some ideas that might be helpful when you 
plan the course schedule. The content you 
will find from your own experience, combined 
with the contents of this manual. 


the course should be 

or a combination of 
theory and practice. The combination is 
strongly recommended, as people have a 
tendency to disbelieve or doubt what you 
say in a classroom situation, while if 
they experience people's responses’ to 
pictures they thought were clearly 
understood by everybody, they may become 
motivated to adapt the practice for their 
projects. 


1. Decide whether 
theoretical only, 


BZ 
ZS 
ZB 


l 


4 


Wi 
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2. The above decision is also dependent 
who attends the course: If you have 
group of people who are going to carry o 
actual pretesting, then practice 
essential, and the course should be abo 
three days. If you are running a cour 
for high level decision-makers, they m 
be able to afford only one day. T 
purpose of such a course will be differe 


from the “how to pretest" you would r 
for the other group. Here, we wi 
concentrate on ideals for ue 


theoretical/practical three day course. 
is often possible to combine such cours 
by inviting decision-makers to the la 
day of such a course; give them 
introduction about pretesting and abo 
what you have done in the course, and th 
allowing them to listen to the results | 
the field testing. This kind of discussi 
can lead to plans for further action a 
follow-up to what has been done, and f 
this it can be useful to have t 
decision-makers present. 


(( 


Do you recognise this communication gap? 


A pretesting course 
should include 
actual practice in 
the field. 


An example of an outline for a training Schedule: 
course: Day. 
a) Introduction 
Objective: To give the participants bb) Why pretesting? Why don't people pretest? 
theoretical and practical skills c) Where and with whom to pretest? 
that will enable them to do the 4d) How many to test with? 
following: Identify target groups e) When to pretest? 
for visuals, test different kinds f) Who can pretest? 
of visuals (single pictures and  g) How to pretest 


series of pictures), draw 

conclusions from the testing, and Techniques (see chapter in this 
use these conclusions to improve manual and decide how 
their materials. much detail you want 


to provide) 

Participants: People who are planning and/or 

implementing projects that use Attitudes and behavior 

communication materials, artists, 

trainers who use visuals in their LUNCH 

teaching of field workers. 

Each participant should bring 

communication materials produced 

by their department for 

discussion and testing. 


h) Role-playing, using the visuals’ the 
participants have brought, and also some 
you may have, which demonstrate certain 
points you want to emphasize or make the 
participants aware of. 

- Demonstration by instructors of good and 
bad techniques in pretesting. 
Participants should comment. Discussion. 

- Participants role-playing, using the 


tee materials that will actually: be used in 
the field next day. Comments’ and 
discussion by participants and 
instructor (s) on techniques and 
behavior. 


i) Orientation about field work 


Day II 

Field-day. The day should spent in a 
village (or a community for which the 
communication materials are developed), 


testing the materials with villagers, and, if 
possible, with users. The participants should 
work in pairs Of two - one asking the 
questions, one recording the interview. They 
should change functions so that each one gets 
experience in interviewing as well as 
recording. 

Select a village not more than about an 
hour's drive from your office. Leave early, 
and spend the whole day (e.g. till 6 p.m.). A 
day is a very short experience, and it is 
important to make the most out of it. 

Somebody from your office - or you 
yourself - should go to the village and make 
the necessary arrangements a few days before 
fom go. It is~advisable to select a village 
that is not already swamped with development 
meajyects. Try ~tGtind a village thaty is 
representative of or similar to the kind of 
places where your materials will be used. 
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Day Lit 
a) Report by the different groups — what wer 


the findings? 
b) Discussion of findings 


LUNCH 
c) Implications of findings: What are 
possible "problem-areas" in visual] 


perception among villagers of the type 
that you interviewed? (Be careful about 
drawing conclusions - your sample is 
small, you have only tested for one day. 
But - you can decide on areas that may 
need further investigations, and _ what 
isues and aspects seem to cause particular 
trouble to the respondents). 

d) Improvement of the visuals you have testec 
(you may need to have half an hour's grout 
work on this aspect, followed by a general 


discussion). 
e) Follow-up to the workshop 
ry CLOSING: 


Your workshop will be more useful to the 
participants if you can provide them witt 
this manual, and/or other reading materials 
before the workshop (at least one week ir 
advance). If ~“pecpie get a change. 
familiarize themselves with the ideas anc 
practices you are going to work with, they 
will learn more from the workshop. 

Pretesting workshops should be run with nc 
more than ten participants. The idea is for 
them all to be able to try out in practice 


what they are learning, both ame ehe 
role-plays and in the field. You need at 
least two instructors - especially for the 
field Situation, where supervision is 


necessary. Each group in the field should be 
observed, and you should give them feedback 
on their performance, as well as pick ur 
issues for discussion the next day. If you 
have too many participants, they will get a 
superficial experience of pretesting, and 
they may not follow up the practices in their 
everyday work, because they do not have 
enough confidence that they can do it on 
their own. If you run an intensive course for 


few participants, chances are much higher 
that they will acquire the necessary 
confidence to continue pretesting. 

However, even with a small course, 


follow-up is essential. One possible idea is 
for the course participants to meet e.g. once 
a month to discuss what they have found out, 
compare the findings, and gradually build up 
expertise in the field of pretesting as well 


as in the knowledge of visual perception. 


